Data processing device, chip, board card and data processing method

Abstract

The application discloses a data processing device, a chip, a board card and a data processing method. The data processing device can be included in a combined processing device as a computing device, and the combined processing device can further include an interface device and other processing devices. The computing device interacts with the other processing devices to jointly complete a user-specified computing operation. The combined processing device can further include a storage device connected with the computing device and the other processing devices respectively, for storing data of the computing device and the other processing devices.

Description

Technical Field

[0001] This application generally relates to the field of data processing technology. More specifically, this application relates to a data processing apparatus, chip, circuit board, and data processing method. Background Technology

[0002] With the rapid development of artificial intelligence technology, the training and inference processes of deep learning models involve the processing of massive amounts of data. In parallel computing devices (such as AI acceleration chips and GPUs), it is often necessary to sort large batches of small-scale data. For example, in applications such as recommendation systems and natural language processing, the data to be processed typically exhibits a distribution characteristic of a huge number of batches (e.g., tens of thousands to hundreds of thousands of batches) and a small size of each batch (e.g., hundreds of elements). Sorting, as one of the fundamental tasks in data processing, directly impacts the overall performance of the computational task.

[0003] Currently, for the aforementioned scenarios involving large volumes of small-scale data, existing parallel computing devices typically employ a batch-based sorting approach: multiple batches of data to be sorted are distributed to various processing cores, and each core independently sorts one or more of its assigned batches. However, due to the small amount of data to be sorted in a single operation, the processor needs to frequently call low-level instructions during the sorting process. Each instruction call incurs significant overhead, such as instruction issue latency and pipeline start / stop, which consumes the majority of the execution time, leaving very little actual data computation time. This severely reduces the processor's instruction execution efficiency, resulting in an overall I / O (input / output) bandwidth utilization of only about 2.5%, leading to a substantial waste of computing resources and failing to meet the speed and energy efficiency requirements of high-performance computing scenarios.

[0004] In view of this, there is an urgent need to provide a data processing solution to effectively improve the sorting speed in scenarios with multiple batches of small-scale data, and to improve the processor's computing efficiency and IO bandwidth utilization. Summary of the Invention

[0005] In order to at least solve one or more of the technical problems mentioned above, this application proposes a data processing device, chip, board and data processing method in several aspects.

[0006] In a first aspect, this application provides a data processing apparatus, including a storage unit and a processing unit; wherein the storage unit is configured to store multiple batches of data to be sorted; the processing unit is configured to: concatenate the multiple batches of data to be sorted into a merged data block, and generate a batch identifier corresponding to each data element in the merged data block; perform a first sorting on the merged data block associated with the batch identifier according to the value of the data element in the merged data block, to obtain a first sorting result; and perform a second sorting on the first sorting result according to the batch identifier, wherein the second sorting is a stable sorting, to obtain a second sorting result, wherein the data elements belonging to the same batch in the second sorting result are arranged consecutively and are internally ordered.

[0007] In a second aspect, this application provides a chip including the data processing apparatus described in the first aspect of this application.

[0008] In a third aspect, this application provides a board including the chip described in the second aspect of this application.

[0009] In a fourth aspect, this application provides a data processing method, comprising: concatenating multiple batches of data to be sorted into a merged data block, and generating a batch identifier corresponding to each data element in the merged data block; performing a first sorting on the merged data block associated with the batch identifier according to the values ​​of the data elements in the merged data block, and obtaining a first sorting result; performing a second sorting on the first sorting result according to the batch identifier, wherein the second sorting is a stable sorting, and obtaining a second sorting result, wherein data elements belonging to the same batch in the second sorting result are arranged continuously and are internally ordered.

[0010] Using the data processing apparatus provided above, this embodiment of the application concatenates multiple batches of data to be sorted into a merged data block for centralized processing, and associates each data element with the batch identifier of its respective batch. First, a first sort is performed based on the value of the data element to achieve global order, and then a stable second sort is performed based on the batch identifier. This allows data elements in the same batch to be continuously clustered after sorting and remain internally ordered. This enables the sorting task of multiple batches to be completed at once without the need for independent processing of each batch. It significantly reduces the frequency of calls to sorting instructions (such as vector filtering instructions) during the sorting process, effectively reduces the instruction launch latency in scenarios with small data volumes in a single batch, thereby greatly improving the overall data processing efficiency and IO bandwidth utilization of the processor, and significantly improving the utilization of hardware resources. Attached Figure Description

[0011] The above and other objects, features, and advantages of exemplary embodiments of this application will become readily understood by reading the following detailed description with reference to the accompanying drawings. In the drawings, several embodiments of this application are illustrated by way of example and not limitation, and the same or corresponding reference numerals denote the same or corresponding parts, wherein:

[0012] Figure 1 This diagram shows a structural schematic of a board 10 according to an embodiment of this application; Figure 2 This is a structural diagram showing the combined processing device in chip 101 of this embodiment; Figure 3 A schematic diagram of the internal structure of a computing device 201 according to an embodiment of this application is shown; Figure 4 The internal architecture of the processor core 306 in an embodiment of this application is shown; Figure 5 Exemplary block diagrams of a data processing apparatus according to some embodiments of this application are shown; Figure 6 Exemplary block diagrams of data processing apparatuses according to other embodiments of this application are shown; Figure 7 An exemplary flowchart of a data processing method according to some embodiments of this application is shown. Detailed Implementation

[0013] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0014] It should be understood that the terms "comprising" and "including" used in the specification and claims of this application indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0015] It should also be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application. As used in this specification and claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this specification and claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations.

[0016] As used in this specification and claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if [described condition or event] is detected" may be interpreted, depending on the context, as "once determined," "in response to determination," "once [described condition or event] is detected," or "in response to detection of [described condition or event]."

[0017] The specific embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0018] Exemplary hardware architecture Figure 1 This diagram illustrates the structure of a board 10 according to an embodiment of this application. Figure 1 As shown, board 10 includes chip 101, which is a system-on-chip (SoC) integrating one or more combined processing units. These combined processing units are artificial intelligence computing units used to support various deep learning and machine learning algorithms, meeting the intelligent processing needs of complex scenarios in fields such as computer vision, speech, natural language processing, data mining, and information recommendation. In particular, deep learning technology is widely used in cloud intelligence. A significant characteristic of cloud intelligence applications is the large volume of input data, placing high demands on the platform's storage and computing capabilities. Board 10 in this embodiment is suitable for cloud intelligence applications, possessing massive off-chip storage, on-chip storage, and substantial computing power.

[0019] Chip 101 is connected to external device 103 via external interface device 102. External device 103 may be, for example, a server, computer, camera, monitor, mouse, keyboard, network card, or Wi-Fi interface. Data to be processed can be transmitted from external device 103 to chip 101 via external interface device 102. The calculation results from chip 101 can be transmitted back to external device 103 via external interface device 102. Depending on the application scenario, external interface device 102 may have different interface forms, such as a PCIe interface.

[0020] The board 10 also includes a storage device 104 for storing data, which includes one or more memory cells 105. The storage device 104 is connected to and transmits data with the controller 106 and the chip 101 via a bus. The controller 106 in the board 10 is configured to regulate the state of the chip 101. Therefore, in one application scenario, the controller 106 may include a microcontroller (MCU).

[0021] Figure 2 This is a structural diagram illustrating the combined processing device in chip 101 of this embodiment. (As shown...) Figure 2 As shown, the combined processing device 20 includes a computing device 201, an interface device 202, a processing device 203, and a DRAM (Dynamic Random Access Memory) 204.

[0022] The computing device 201 is configured to perform user-specified operations. It is mainly implemented as a single-core intelligent processor or a multi-core intelligent processor to perform deep learning or machine learning calculations. It can interact with the processing device 203 through the interface device 202 to jointly complete the user-specified operations.

[0023] Interface device 202 is used to transmit data and control commands between computing device 201 and processing device 203. For example, computing device 201 can obtain input data from processing device 203 via interface device 202 and write it to on-chip storage device of computing device 201. Further, computing device 201 can obtain control commands from processing device 203 via interface device 202 and write them to on-chip control cache of computing device 201. Alternatively or optionally, interface device 202 can also read data from storage device of computing device 201 and transmit it to processing device 203.

[0024] The processing device 203, as a general-purpose processing device, performs basic controls including but not limited to data transfer and starting / stopping the computing device 201. Depending on the implementation, the processing device 203 may be one or more types of processors, such as a central processing unit (CPU), a graphics processing unit (GPU), or other general-purpose and / or special-purpose processors. These processors include, but are not limited to, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., and their number can be determined according to actual needs. As mentioned above, the computing device 201 of this application can be considered as having a single-core structure or a homogeneous multi-core structure. However, when the computing device 201 and the processing device 203 are considered together, they are considered to form a heterogeneous multi-core structure.

[0025] DRAM 204 is used to store data to be processed. It can be DDR (Double Data Rate Synchronous Dynamic Random Access Memory) memory, typically 16G or larger, used to store data of computing device 201 and / or processing device 203.

[0026] Figure 3 A schematic diagram of the internal structure of a computing device 201 according to an embodiment of this application is shown. The computing device 201 is used to process input data such as computer vision, speech, natural language processing, data mining, and information recommendation. The computing device 201 in the figure adopts a multi-core hierarchical structure design. As a system-on-a-chip, the computing device 201 includes multiple computing clusters, and each computing cluster includes multiple processor cores. In other words, the computing device 201 is a system-on-a-chip... Computational clusters The processor cores are structured in layers.

[0027] From the perspective of system-on-a-chip hierarchy, such as Figure 3 As shown, the computing device 201 includes an external storage controller 301, a peripheral communication module 302, an on-chip interconnect module 303, a synchronization module 304, and multiple computing clusters 305.

[0028] There can be multiple external storage controllers 301; two are shown as an example in the figure. These controllers are used to access external storage devices, such as those issued by the processor core, in response to access requests from the processor core. Figure 2 The DRAM 204 in the chip allows the computing device 201 to read data from external sources or write data to external sources. The peripheral communication module 302 receives control signals from the processing device 203 via the interface device 202, initiating the computing device 201 to execute tasks. The on-chip interconnect module 303 connects the external storage controller 301, the peripheral communication module 302, and multiple computing clusters 305 to transmit data and control signals between modules. The synchronization module 304 is a global barrier controller (GBC) used to coordinate the working progress of each computing cluster and ensure information synchronization. The multiple computing clusters 305 are the computing core of the computing device 201. Four are shown exemplarily in the figure; however, with hardware development, the computing device 201 of this application may also include 8, 16, 64, or even more computing clusters 305. The computing clusters 305 are used to efficiently execute deep learning algorithms.

[0029] From the perspective of the hierarchy of computational clusters, such as Figure 3 As shown, each computing cluster 305 includes multiple processor cores (IPU cores) 306 and one memory core (MEM core) 307.

[0030] Four processor cores 306 are shown in the figure as an example, but this application does not limit the number of processor cores 306. Its internal architecture is as follows: Figure 4 As shown. Each processor core 306 includes three main modules: a control module 41, an arithmetic module 42, and a storage module 43.

[0031] The control module 41 coordinates and controls the operation of the computation module 42 and the storage module 43 to complete the deep learning task. It includes an instruction fetch unit (IFU) 411 and an instruction decode unit (IDU) 412. The instruction fetch unit 411 fetches instructions from the processing device 203, and the instruction decode unit 412 decodes the fetched instructions and sends the decoding result as control information to the computation module 42 and the storage module 43.

[0032] The computation module 42 includes a vector operation unit 421 and a matrix operation unit 422. The vector operation unit 421 is used to perform vector operations and can support complex operations such as vector multiplication, addition, and nonlinear transformations; the matrix operation unit 422 is responsible for the core computations of deep learning algorithms, namely matrix multiplication and convolution.

[0033] Storage module 43 is used to store or move relevant data, including neuron RAM (NRAM) 431, weight RAM (WRAM) 432, input / output direct memory access (IODMA) 433, and move direct memory access (MVDMA) 434. NRAM 431 stores input, output data, and intermediate results for computation by processor core 306; WRAM 432 stores the weights of the deep learning network; IODMA 433 controls memory access between NRAM 431 / WRAM 432 and DRAM 204 via broadcast bus 309; MVDMA 434 controls memory access between NRAM 431 / WRAM 432 and SRAM 308. It should be noted that NRAM and WRAM here can be two storage regions formed by dividing the same memory in logical storage space, or they can be two independent memories; no specific limitation is made here.

[0034] Back Figure 3The storage core 307 is primarily used for storage and communication, namely storing shared data or intermediate results among processor cores 306, and performing communication between computing clusters 305 and DRAM 204, communication between computing clusters 305, and communication between processor cores 306. In other embodiments, the storage core 307 has scalar operation capabilities and is used to perform scalar operations.

[0035] Storage core 307 includes shared memory unit (SRAM) 308, broadcast bus 309, cluster direct memory access (CDMA) module 310, and global direct memory access (GDMA) module 311. SRAM 308 acts as a high-performance data relay station. Data multiplexed between different processor cores 306 within the same computing cluster 305 does not need to be obtained from DRAM 204 by each processor core 306. Instead, it is relayed between processor cores 306 via SRAM 308. Storage core 307 only needs to quickly distribute the multiplexed data from SRAM 308 to multiple processor cores 306 to improve inter-core communication efficiency and greatly reduce on-chip and off-chip input / output access.

[0036] Broadcast bus 309, CDMA 310, and GDMA 311 are used to perform communication between processor cores 306, communication between computing clusters 305, and data transfer between computing clusters 305 and DRAM 204, respectively. These will be explained separately below.

[0037] The broadcast bus 309 is used to complete high-speed communication between the processor cores 306 within the computing cluster 305. In this embodiment, the broadcast bus 309 supports inter-core communication methods including unicast, multicast, and broadcast. Unicast refers to point-to-point (i.e., data transmission from one processor core to another) data transmission. Multicast is a communication method that transmits a piece of data from SRAM 308 to several specific processor cores 306. Broadcast is a communication method that transmits a piece of data from SRAM 308 to all processor cores 306, and is a special case of multicast.

[0038] CDMA 310 is used to control memory access of SRAM 308 between different computing clusters 305 within the same computing device 201. GDMA 311 works in conjunction with the external memory controller 301 to control memory access from SRAM 308 of computing cluster 305 to DRAM 204, or to read data from DRAM 204 into SRAM 308. As described above, communication between DRAM 204 and NRAM 431 or WRAM 432 can be achieved through two channels. The first channel is through IODAM 433 directly connecting DRAM 204 and NRAM 431 or WRAM 432; the second channel is to first transmit data between DRAM 204 and SRAM 308 via GDMA 311, and then transmit data between SRAM 308 and NRAM 431 or WRAM 432 via MVDMA 434. Although the second channel appears to require more components and has a longer data flow, in some embodiments, the bandwidth of the second channel is actually much greater than that of the first channel. Therefore, communication between DRAM 204 and NRAM 431 or WRAM 432 may be more efficient through the second channel. Embodiments of this application may select the data transmission channel based on their own hardware capabilities.

[0039] In other embodiments, the functions of GDMA 311 and IODMA 433 can be integrated into the same component. For ease of description, this application treats GDMA 311 and IODMA 433 as different components. For those skilled in the art, any component whose implemented functions and achieved technical effects are similar to those of this application falls within the scope of protection of this application. Furthermore, the functions of GDMA 311, IODMA 433, CDMA 310, and MVDMA 434 can also be implemented by the same component. Similarly, any component whose implemented functions and achieved technical effects are similar to those of this application falls within the scope of protection of this application.

[0040] Exemplary data processing device Figure 5 Exemplary block diagrams of a data processing apparatus according to some embodiments of this application are shown. Figure 5As shown, the data processing device 500 includes a storage unit 510 and a processing unit 520; wherein, the storage unit 510 is configured to store multiple batches of data to be sorted; the processing unit 520 is configured to: concatenate the multiple batches of data to be sorted into a merged data block, and generate a batch identifier corresponding to each data element in the merged data block; perform a first sorting on the merged data block associated with the batch identifier according to the value of the data element in the merged data block, and obtain a first sorting result; perform a second sorting on the first sorting result according to the batch identifier, and the second sorting is a stable sorting, and obtain a second sorting result, wherein the data elements belonging to the same batch in the second sorting result are arranged continuously and are internally ordered.

[0041] In some implementations, the data processing device 500 can be implemented as described above, combining... Figures 3-4 The processor core 306 is described. In this embodiment, the storage unit 510 may be a private on-chip cache within the data processing device 500, for example... Figure 4 The NRAM (Neural RAM) or L1 Data Cache shown is used to temporarily store data to be processed and intermediate results during the sorting process. Storage unit 510 can load appropriate batches of data to be sorted from external storage units for sorting processing by the arithmetic unit 520 in the data processing device 500. The arithmetic unit 520, as the core processing component of the data processing device 500, can be implemented as follows: Figure 4 The computation module or vector operation unit shown is used to perform computational tasks such as data comparison, moving, and compression writing in sorting operations.

[0042] In applications such as recommender systems and natural language processing, data to be sorted is typically organized in batches, meaning it contains multiple batches, each with a certain number of data elements. These data elements can be feature values, scores, or other numerical data requiring sorting. Since the data size of a single batch is usually small (e.g., hundreds of elements), and the storage unit 510, as an on-chip cache, has a limited capacity but is sufficient to hold multiple batches of data and corresponding auxiliary information, the processing unit 520 is configured to perform batch concatenation operations: retrieving multiple batches of data to be sorted from the storage unit 510 or external storage units, and concatenating these logically independent and potentially physically scattered batches into a single merged data block. The concatenation can be achieved, for example, by allocating contiguous storage address space for each batch of data within the storage unit 510, arranging the data in a preset order to achieve physical contiguous storage. Simultaneously, the processing unit 520 also generates a batch ID corresponding to each data element in the merged data block. This batch ID indicates the original batch to which each data element belongs; essentially, it is a tag attached to each data element. Through the above concatenation and ID generation operations, the originally discrete multiple independent sorting tasks are transformed into a unified sorting task, providing a data foundation for subsequent centralized processing.

[0043] It should be noted that the merged data block associated with batch identifiers mentioned above refers to a binding relationship between each data element in the merged data block and its corresponding batch identifier. When a data element moves during the sorting process, its corresponding batch identifier moves synchronously with the data element. This binding relationship can be implemented in various ways. For example, data elements and their corresponding batch identifiers can be configured as key-value pairs to participate in the sorting together; or two parallel arrays can be maintained for each data element and the batch identifier, and they can be synchronously exchanged according to the same index; or data elements and batch identifiers can be encapsulated into composite data types such as structures for unified processing. Regardless of the specific binding method used, the core purpose is to ensure that the correspondence between data elements and their batch affiliation is maintained throughout the entire sorting process. The affiliation information will not be lost or disordered due to the movement of data elements in storage unit 510, thus ensuring that each data element can still be accurately traced back to its original batch after sorting.

[0044] Based on this, the operation unit 520 further performs a first sorting operation on the merged data blocks associated with batch identifiers. This first sorting can use the values ​​of data elements in the merged data blocks as sorting keys. After the first sorting, all data elements in the merged data blocks are globally ordered according to their numerical values. However, since the data from different batches were mixed during the concatenation process, the data elements from different batches are interleaved in the first sorting result and have not yet been distinguished by batch.

[0045] In some embodiments, the first sorting can be implemented using any sorting algorithm suitable for parallel computing devices, such as radix sort or other sorting methods that support bit-by-bit distribution, as long as it can globally rearrange the data elements in the merged data block. When the first sorting is implemented using radix sort, since radix sort is a non-comparative sorting algorithm based on bit-by-bit distribution, the operation unit 520 can achieve efficient parallel processing by calling the underlying vector filter instruction. This vector filter instruction can process the current bit of all data elements in the merged data block in parallel within a single instruction cycle, and compress and write the data elements along with their corresponding batch identifiers into the contiguous storage space in the storage unit 510 according to the comparison results, thereby fully utilizing the data parallel processing capability of the vector processing unit.

[0046] After the first sorting is completed, the operation unit 520 also performs a second sorting operation, which can use the batch identifier as the sorting key to re-sort the results of the first sorting. The key constraint of the second sorting is that it must be a stable sort. A stable sort means that when the sorting key (in this case, the batch identifier) ​​is the same, the relative order of these elements with the same key remains unchanged before and after sorting. The above-mentioned stability constraint plays a key role in the technical solution of this application: since the first sorting has already made the merged data blocks globally ordered by data value, the second sorting, when sorting by the batch identifier as the key, ensures that data elements belonging to the same batch can be grouped together to form continuous data blocks after sorting, while maintaining the relative order established by the first sorting, that is, arranged in order of data value. If the second sorting is not stable, the relative order between data elements within the same batch may be disrupted, thereby destroying the effect of the first sorting. In some embodiments, the second sorting can also be implemented by calling the above-mentioned vector filtering instruction, and its sorting rounds depend on the effective binary bits (or, the required binary bits) of the batch identifier. For example, when the number of batches to be merged is 50, the effective binary bits of the batch identifier are 6 bits. Therefore, only 6 rounds of second sorting are needed, which is much less than the number of rounds required for the first sorting (e.g., 32 bits of data need to be processed), thus ensuring the overall sorting efficiency.

[0047] After the two rounds of sorting, the resulting second sorting order in storage unit 510 exhibits a clear data structure: data elements belonging to the same original batch are arranged consecutively in the sorting result, and within each batch, the data elements are ordered according to their values. In other words, by employing a "global sorting by data value followed by stable clustering by batch identifier" approach, the computation unit 520 completes multiple batch sorting tasks on-chip at once without relying on independent batch sorting. Furthermore, the sorting results of each batch are logically independent and naturally separated by continuous address ranges. This process integrates sorting tasks that originally required multiple independent schedulings into a single centralized processing flow, eliminating the high-frequency instruction call overhead of batch processing. This allows for full utilization of the on-chip storage resources of storage unit 510, significantly improving the sorting efficiency of the data processing device in scenarios with small batch data sizes.

[0048] Figure 5The data processing device 500 shown constructs a multi-batch data processing architecture through the coordinated operation of the storage unit 510 and the computing unit 520, which first performs centralized sorting and then clusters data by affiliation. This data processing device utilizes its internal on-chip storage resources as the main storage space for sorting operations. At the storage level, it concatenates multiple batches of data to be sorted into merged data blocks. At the computing level, it employs a two-step sorting strategy of numerical sorting and stable clustering, enabling the sorting tasks of multiple batches to be completed centrally within the same processor core. This ensures both the correctness and independence of the sorting results for each batch and fully utilizes the vector processing capabilities of the processor core and the high bandwidth of the on-chip storage. Since each data processing device 500 can be implemented as an independent processor core, a chip containing multiple such data processing devices can process different batches of sorting tasks in parallel, thereby achieving higher processing throughput at the overall system level and effectively meeting the requirements of high-performance computing scenarios for data processing speed and energy efficiency.

[0049] In some embodiments, both the first and second sorting methods described above can be implemented using Least Significant Digit (LSD) radix sort. The core idea of ​​radix sort is to distribute and collect the data to be sorted bit by bit according to its binary representation, iterating multiple times until the overall data reaches an ordered state. Least Significant Digit (LSD) radix sort starts from the least significant bit (i.e., bit 0) and sorts and distributes the data bit by bit in successive rounds. Taking a 32-bit data type as an example, LSD radix sort needs to process bits 0 to 31 sequentially, performing a total of 32 rounds of distribution operations.

[0050] Radix sort was chosen as the preferred sorting implementation in this application because its bit-by-bit distribution mechanism is highly compatible with the vector filtering instructions provided by the underlying parallel computing device. Vector filtering instructions are hardware primitives that perform parallel comparison and compressed writing of data elements according to their binary bits. By adopting least-bit-first (MSD) radix sort, the inherent stability of MSD in standard implementations can be leveraged, providing an algorithmic guarantee for the stability required in the second sorting. This allows the same basic sorting logic to be reused directly in this embodiment without introducing an additional stable sorting guarantee mechanism.

[0051] In some embodiments, the operation unit 520 is further configured to: during the first sorting, distribute data elements bit by bit according to their binary width; and during the second sorting, distribute data elements bit by bit according to the number of binary bits required for the batch identifier. For ease of explanation, taking a data element represented as a 32-bit floating-point number or a 32-bit integer as an example, its binary width is 32 bits. Therefore, the first sorting requires 32 rounds of bit-by-bit distribution until all bits are processed, ensuring the merged data block is completely ordered globally based on the data element values. In contrast, the second sorting targets batch identifiers, whose value range depends on the number of batches N concatenated into the merged data block. Since the batch identifier only needs to distinguish N different original batches, the required number of binary bits is much smaller than the width of the data element itself. For example, with N=50, the batch identifier ranges from 0 to 49, requiring only 6 binary bits to represent it completely. Therefore, the second sorting only requires 6 rounds of bit-by-bit distribution to complete the sorting and clustering by batch identifier. With the above configuration, the second sorting rounds of this application are much fewer than the first sorting rounds, which further reduces the computational overhead while ensuring the correctness of the sorting. This is because the effective number of bits of the batch identifier is determined by the number of batches N, and N is limited by the available capacity of the on-chip storage unit, thus making the second sorting more efficient than the first sorting.

[0052] Furthermore, when the number of batches concatenated into the merged data block is N, the number of binary bits required for the batch identifier can be ceil(log₂N), where ceil() represents rounding up. Since N different batches require N different identifier values ​​to distinguish them, and M binary bits can represent at most 2^N... M There are 2 distinct values, therefore the minimum number of binary bits required to make 2 M The smallest integer M that is greater than or equal to N. Therefore, the number of binary bits required for the batch identifier can be represented as ceil(log₂N), which is the floor function of the logarithm of N to the base 2.

[0053] For example, when N=3, log₂3≈1.58, ceil(1.58)=2, meaning 2 bits are sufficient to distinguish 3 different batches; when N=16, log₂16=4, ceil(4)=4, meaning 4 bits are just enough to distinguish 16 different batches. Therefore, the number of sorting rounds required for the second sorting is ceil(log₂N), which increases logarithmically with the number of batches N, rather than linearly. This ensures that even in scenarios requiring the merging of many batches, the additional overhead of the second sorting remains low and does not significantly impact the overall sorting efficiency.

[0054] In some embodiments, the batch identifier can be stored in storage unit 510 with the required number of binary bits. In other preferred embodiments, although the batch identifier logically requires only ceil(log₂N) binary bits for differentiation, to simplify the data organization in storage unit 510 and improve data access efficiency, the batch identifier can be stored in storage unit 510 with the same binary bit width as the data element. For example, when the data element is a 32-bit floating-point number, each batch identifier also occupies 32 bits of storage space. Although this storage method increases the storage capacity occupied by the batch identifier to some extent, its engineering advantages are significant. On the one hand, using the same data bit width as the data element avoids data type conversion and bit width alignment processing during storage and loading, simplifying the read and write control logic of storage unit 510; on the other hand, a unified data bit width allows data elements and batch identifiers to be accessed with the same address step, which is beneficial to improving the access efficiency of the storage unit.

[0055] It's important to note that the storage width of the batch identifier and the number of bits required for sorting are two different concepts: the storage width refers to the physical storage space occupied, while the number of bits required for sorting refers to the logical number of binary bits involved in the sorting. In other words, even if the batch identifier is stored with a 32-bit storage width, the arithmetic unit 520 can still process only its lower ceil (log₂N) significant bits when performing the second sorting. The higher zero-padding bits do not need to participate in the sorting. Therefore, it not only does not affect the execution efficiency of the second sorting but also achieves greater benefits such as simplified read / write control logic and improved access efficiency.

[0056] In some embodiments, after obtaining the second sorting result, the operation unit 520 is further configured to divide the second sorting result according to the original data size of each batch (i.e., the number of data elements contained in each batch), and output the divided second sorting result to an off-chip storage unit. Since the data elements belonging to the same batch in the second sorting result are arranged consecutively, and each batch contains a fixed number of data elements, the operation unit 520 only needs to use the number of elements in each batch as a fixed address offset to accurately divide the start address and end address of each batch from the merged data block of the second sorting result, thereby obtaining the independent sorting result of each batch. Taking a batch containing 448 data elements as an example, the elements from 0 to 447 in the second sorting result are the sorting result of batch 0, the elements from 448 to 895 are the sorting result of batch 1, and so on.

[0057] The above-described segmentation operation requires no additional searching, marking, or moving of data; it can be completed solely through address offset calculation and memory block copying, resulting in extremely low computational overhead. After segmentation, the arithmetic unit 520 outputs the sorting results of each batch from the storage unit 510 (on-chip storage) to an off-chip storage unit (e.g., global memory, such as high-bandwidth memory HBM or the aforementioned combination). Figures 2-4 The described DRAM is used for subsequent computational tasks (such as Top-K selection, quantile calculation, or feature selection). Through the above splitting and output operations, the multiple batch sorting results originally intertwined in the merged data blocks are restored to independent batch sorting results, which are stored in off-chip storage units with the same batch organization structure as the input data, achieving seamless connection with downstream data processing flows.

[0058] To better understand the specific implementation process of the technical solutions in the embodiments of this application, the following example illustrates the process using two batches, each with three data elements.

[0059] Suppose that storage unit 510 stores two batches of data to be sorted, where the data elements of the first batch (batch 0) are [30, 10, 20] and the data elements of the second batch (batch 1) are [25, 5, 15]. The objective of this application is to sort the data of the two batches respectively, so that batch 0 is sorted as [10, 20, 30] and batch 1 is sorted as [5, 15, 25].

[0060] First, as shown in Table 1, the operation unit 520 concatenates the data from the two batches into a merged data block and generates a batch identifier corresponding to each data element in the merged data block. The concatenated merged data block stores six data elements sequentially in the storage unit 510, in the following order: 30 (batch 0), 10 (batch 0), 20 (batch 0), 25 (batch 1), 5 (batch 1), 15 (batch 1). Simultaneously, the generated batch identifier tensor corresponds one-to-one with each data element in the merged data block, in the following order: 0, 0, 0, 1, 1, 1, indicating the original batch to which each data element belongs. At this point, the data elements in the merged data block have formed a continuous physical storage layout, and the batch identifier is stored along with it as an associated attribute of each data element.

[0061] Table 1: Merged data blocks associated with batch identifiers

[0062] Subsequently, the operation unit 520 uses the values ​​of each data element in the merged data block as the first key to perform a first sort on the merged data block. In this embodiment, the first sort is implemented using a least-bit-first radix sort. During the sorting and distribution, the data elements and their corresponding batch identifiers are moved as a whole. That is, when a data element needs to be moved due to sorting, its corresponding batch identifier moves synchronously to the same position along with the data element. After the first sort is completed, the six data elements in the merged data block are globally ordered according to their numerical values. The sorted state is shown in Table 2, where the data elements are arranged as [5,10,15,20,25,30], and their corresponding batch identifier sequence becomes [1,0,1,0,1,0]. At this time, although all data elements are arranged neatly according to their numerical values, elements from different batches are interspersed in the merged data block and have not yet been distinguished according to batch.

[0063] Table 2: First Sorting Results

[0064] Next, the operation unit 520 uses the batch identifier as the second key to perform a second sort on the first sorting result. The second sort also uses radix sort, and it must be a stable sort. Since radix sort has inherent stability under the least significant bit-first standard implementation, it can directly reuse the same basic sorting logic as the first sort, only requiring the sorting key to be changed from the data element value to the batch identifier. During the second sorting process, the operation unit 520 rearranges the data elements according to the value of the batch identifier. Due to stability constraints, the relative order between data elements belonging to the same batch is completely preserved. That is, data elements with batch identifier 0 are arranged according to their original order of appearance in the first sorting result, and data elements with batch identifier 1 are arranged according to their original order of appearance in the first sorting result. The second sorting result is shown in Table 3.

[0065] Table 3: Second Sorting Results

[0066] As shown in Table 3, after the second sorting is completed, the data elements in the merged data block are arranged in the order of [10,20,30,5,15,25], and the corresponding batch identifier sequence is [0,0,0,1,1,1]. It can be clearly verified that data elements belonging to the same batch in the second sorting result are arranged consecutively: indices 0 to 2 correspond to batch 0, and indices 3 to 5 correspond to batch 1; furthermore, within each batch, the data elements are ordered according to their values: the internal order of batch 0 is [10,20,30], and the internal order of batch 1 is [5,15,25]. Thus, the sorting tasks for both batches have been completed centrally in the data processing device 500 in one go. The resulting second sorting result maintains both continuous clustering between batches and ensures order within each batch.

[0067] Although the above example illustrates two batches with three data elements per batch, it is understandable that in real-world applications, the number of batches can be greater, and the number of data elements per batch can be even greater.

[0068] Figure 6 Exemplary block diagrams of data processing apparatuses according to other embodiments of this application are shown. For example... Figure 6 As shown, the data processing device 500 includes a storage unit 510 and a processing unit 520, and also includes a control unit 530, wherein the control unit 530 can be configured to issue vector filtering instructions to the processing unit 520 to control the processing unit 520 to execute the first sorting and the second sorting in sequence.

[0069] As mentioned earlier, vector filtering instructions are a hardware primitive provided at the underlying level of parallel computing devices, capable of performing parallel comparison and compressed writing of all data elements in the merged data block stored in memory unit 510 within a single instruction cycle. The control unit 530 serves as the instruction control logic of the processor core (which can be implemented, for example, in combination with the previously mentioned...). Figure 4 The control module 41 described is responsible for decoding and scheduling the vector filtering instructions. It sequentially issues vector filtering instructions for each round according to the required sorting operation flow, driving the arithmetic unit 520 to complete the bit-by-bit sorting and distribution operation. By configuring a dedicated control unit 530 in the data processing device 500 to manage the timing and parameter configuration of the vector filtering instructions, the execution processes of the first and second sorting can be precisely controlled by the hardware instruction stream. This further reduces the proportion of control overhead during instruction execution, enabling the arithmetic unit 520 to continuously execute the sorting task with maximum efficiency.

[0070] In some embodiments, the number N of batches concatenated into the merged data block can be determined as needed. In other preferred embodiments, the number N of batches concatenated into the merged data block can be determined based on the available capacity of the storage unit 510. For example, the control unit 530 can also be configured to determine the number of batches of data to be sorted in the merged data block based on the available capacity of the storage unit 510. The specific determination method is as follows: Since each data element in the merged data block corresponds to a batch identifier, the sum of the storage space occupied by the merged data block and the batch identifier must not exceed the available capacity of the storage unit 510; otherwise, it will be impossible to completely store all the data required for sorting in the on-chip storage. Therefore, before performing the sorting task, the control unit 530 can first obtain the current available capacity of the storage unit 510, and calculate the maximum number of batches N that can be accommodated without exceeding the available capacity, based on the data size of each batch and the storage space occupied by the batch identifier corresponding to each data element. This number is then used as the actual number of batches concatenated for this sorting process.

[0071] For example, assuming that the available capacity of storage unit 510 is 224KB, each batch contains 448 data elements, and each data element and its corresponding batch identifier each occupy 4 bytes (32 bits), then the storage space required for each batch is 448×4×2=3584 bytes. Therefore, the maximum number of batches that can be merged can be calculated to be approximately 224KB÷3584≈64 batches.

[0072] In this way, the control unit 530 can adaptively determine the optimal batch number N before each sorting operation begins, which makes full use of on-chip storage resources and avoids sorting failure or performance degradation caused by data overflow.

[0073] As described in the previous embodiments, the number of binary bits required for the batch identifier in the second sort is ceil(log2N), while the binary bit width of the data elements in the first sort is determined by the data type. Therefore, during the execution of a complete sorting task in one processing cycle (i.e., the first sort and the second sort), the total number of vector filtering instructions issued by the control unit 530 can be equal to the sum of the binary bit width of the data elements in the first sort and the number of binary bits required for the batch identifier in the second sort.

[0074] Taking a data element as a 32-bit binary number and a batch size of N=50 as an example, the first sorting requires processing 32 bits one by one, and the second sorting requires processing 6 bits one by one (because ceil(log250)=6). The control unit 530 issues a total of 32+6=38 vector filtering instructions to complete all sorting.

[0075] The above quantitative relationship clearly shows that compared to the previous batch processing method, where each batch required 32 vector filtering instructions and processing N batches individually required a total of N×32 instructions, this embodiment reduces the number of instructions issued from N×32 to 32+ceil(log2N) by merging multiple batches for a single two-round sorting. For example, with N=50, the number of instructions issued decreases from 1600 to 38, a reduction of approximately 42 times. This order-of-magnitude reduction in instructions directly leads to a significant reduction in instruction issuance latency, freeing the arithmetic unit 520 from the frequent start-stop of the instruction pipeline and allowing it to allocate more computing resources to actual data processing.

[0076] In summary, by introducing a control unit 530 into the data processing device 500, this embodiment of the application achieves centralized scheduling and precise control of vector filtering instructions. On the one hand, the control unit 530 dynamically determines the number of merging batches N based on the available capacity of the on-chip storage unit 510, ensuring that the data can be completely sorted in the on-chip storage. On the other hand, through an optimized instruction issuance strategy, the total number of vector filtering instructions required to complete the sorting of all batches is only the sum of the data bit width and the number of bits required for the batch identifier, and is not directly related to the number of batches to be sorted. This feature is particularly crucial in scenarios with a large number of batches. Regardless of how many batches need to be sorted, within a processing cycle of a merged data block, the number of instructions issued by the control unit 530 depends only on the data type bit width and the number of bits required for the batch identifier, fundamentally solving the problem of low hardware efficiency caused by excessive instruction issuance in batch processing.

[0077] To verify the actual effect of the above technical solutions in the embodiments of this application, the inventors conducted experimental verification using a parallel computing device with a single chip containing 16 processing cores as the test environment. In this test scenario, the data to be sorted contained 98,820 batches, each batch containing 448 data elements (taking 32-bit floating-point numbers as an example, the data size of a single batch is 448 × 4 = 1792 bytes, approximately 1.75KB). The 16 processing cores evenly distributed the above batches, with each processor core needing to process approximately 6176 batches.

[0078] In the existing scheme, each processor core performs sorting independently for each batch, processing only 1.75KB of data per batch. This requires 32 vector filtering instruction calls, totaling 6176 × 32 = 197632 vector filtering instruction calls to complete the sorting of all 6176 batches. Because the amount of data sorted per batch is extremely small, the fixed overhead of instruction issue delay and pipeline start / stop during each instruction call dominates, resulting in a significant amount of processor time being consumed by instruction scheduling rather than actual data computation. However, using the scheme of this application, taking a single merge of 50 batches as an example, each merged data block contains 50 × 448 = 22400 data elements, approximately 87.5KB in size. The first sorting requires processing 32 bits of 32-bit data elements, and the second sorting requires processing ceil(log250) = 6 bits of the batch identifier. Completing the entire sorting of one merged data block requires only 32 + 6 = 38 vector filtering instruction calls. The 6,176 batches that each processor core needs to process are divided into approximately 124 merged data blocks (6,176 ÷ 50 ≈ 124), requiring a total of only 124 × 38 = 4,712 vector filtering instruction calls, which is about 42 times less than the 197,632 calls required by the existing scheme.

[0079] The reduction in the number of instruction calls directly translates to an improvement in I / O bandwidth utilization. I / O bandwidth utilization is defined as the ratio of theoretical I / O time to actual time consumption (including actual I / O time and actual computation time). Theoretical I / O time is equal to the total amount of data to be sorted divided by the hardware peak bandwidth, i.e., considering only the ideal shortest time required for data movement. Regardless of whether the existing scheme or the scheme in this application is used, the total amount of data that each processing core needs to sort remains unchanged (both are 6176 × 448 × 4 ≈ 11 MB), therefore the theoretical I / O time is exactly the same in both schemes.

[0080] However, the composition of actual time consumption varies significantly across different schemes: In existing schemes, due to the extremely small amount of data sorted per operation, the actual time consumption is dominated by instruction issuance latency. Both actual I / O time and computation time are hampered by the low efficiency of the instruction pipeline, resulting in actual time consumption far exceeding theoretical I / O time, with measured I / O bandwidth utilization of only about 2.5%. In contrast, by adopting the scheme of this application, the number of instruction calls is reduced by 42 times, the proportion of instruction issuance latency in the total time consumption decreases significantly, and the actual total time consumption is significantly shortened, approaching the theoretical I / O time. Therefore, the I / O bandwidth utilization is increased to about 7.1%. Dividing the two, the reduction factor of the actual total time consumption is 2.86 times.

[0081] The above-mentioned measured data fully verifies the actual effect of the technical solution of this application embodiment in typical application scenarios. By merging multiple batches and sorting them uniformly, and distributing them bit by bit based on vector filtering instructions, this application embodiment reduces the number of vector filtering instruction calls required for each processing core from 197,632 to approximately 4,712, and increases the IO bandwidth utilization from approximately 2.5% to approximately 7.1%, while ensuring the correctness of the sorting results, resulting in an overall performance improvement of approximately 2.84 times. The above performance improvement has significant application value in parallel computing scenarios with a large number of batches and a small amount of data per batch, effectively releasing the computing potential of the processor and significantly improving hardware efficiency.

[0082] This application also provides a chip that may include the data processing apparatus of any of the embodiments described above in conjunction with the accompanying drawings. The chip may include one or more data processing apparatuses.

[0083] Furthermore, this application embodiment also provides a board card, which may include the aforementioned chip.

[0084] Exemplary data processing method Figure 7 Exemplary flowcharts of data processing methods according to some embodiments of this application are shown. Figure 7 As shown, the data processing method 700 includes: step S702, concatenating multiple batches of data to be sorted into a merged data block, and generating a batch identifier corresponding to each data element in the merged data block; step S704, performing a first sort on the merged data block associated with the batch identifier according to the value of the data element in the merged data block, and obtaining a first sorting result; step S706, performing a second sort on the first sorting result according to the batch identifier, wherein the second sort is a stable sort, and obtaining a second sorting result, wherein the data elements belonging to the same batch in the second sorting result are arranged continuously and are internally ordered.

[0085] In some embodiments, both the first sorting and the second sorting are radix sorting with least significant digit priority.

[0086] In other embodiments, the data processing method 700 further includes: when performing a first sort, invoking a vector filtering instruction to sort and distribute data elements bit by bit according to the binary bit width of the data elements; and when performing a second sort, invoking a vector filtering instruction to sort and distribute data elements bit by bit according to the number of binary bits required for the batch identifier.

[0087] In some other embodiments, the number of binary bits required for the batch identifier is ceil(log2N), where N is the number of batches concatenated into the merged data block.

[0088] In some embodiments, the batch identifier is represented using the same binary bit width as the data element.

[0089] In other embodiments, within a processing cycle of a merged data block, the total number of times the vector filtering instruction is invoked is equal to the sum of the binary bit width of the data elements in the first sort and the number of binary bits required for the batch identifier in the second sort.

[0090] In some other embodiments, the data processing method 700 may further include: dividing the second sorting result according to the original data size of each batch to obtain an independent sorting result for each batch.

[0091] Those skilled in the art will understand that the description of the data processing apparatus of the present application in conjunction with the accompanying drawings can also be applied to the data processing method of the present application, and therefore will not be repeated.

[0092] The foregoing can be better understood in accordance with the following terms: Clause A1. A data processing apparatus, comprising a storage unit and a processing unit; wherein the storage unit is configured to store multiple batches of data to be sorted; the processing unit is configured to: concatenate the multiple batches of data to be sorted into a merged data block, and generate a batch identifier corresponding to each data element in the merged data block; perform a first sorting on the merged data block associated with the batch identifier according to the value of the data element in the merged data block, to obtain a first sorting result; perform a second sorting on the first sorting result according to the batch identifier, wherein the second sorting is a stable sorting, to obtain a second sorting result, wherein data elements belonging to the same batch in the second sorting result are arranged consecutively and are internally ordered.

[0093] Clause A2, the data processing apparatus according to Clause A1, wherein both the first sort and the second sort are least significant digit-first radix sorts.

[0094] Clause A3, the data processing apparatus according to Clause A1 or A2, further includes: a control unit configured to issue a vector filtering instruction to the arithmetic unit to control the arithmetic unit to sequentially execute the first sorting and the second sorting.

[0095] Clause A4. The data processing apparatus according to Clause A3, wherein the control unit is further configured to: determine the number of batches of data to be sorted and concatenated into the merged data block based on the available capacity of the storage unit, wherein the sum of the storage space occupied by the merged data block and the batch identifier does not exceed the available capacity of the storage unit.

[0096] Clause A5. The data processing apparatus according to Clause A1, wherein the arithmetic unit is further configured to: when performing the first sorting, sort and distribute the data elements bit by bit according to the binary bit width of the data elements; and when performing the second sorting, sort and distribute the data elements bit by bit according to the number of binary bits required for the batch identifier.

[0097] Clause A6, the data processing apparatus according to Clause A5, wherein the number of binary bits required for the batch identifier is ceil(log2N), where N is the number of batches concatenated to the merged data block.

[0098] Clause A7, the data processing apparatus according to Clause A5 or A6, wherein the batch identifier is stored in the storage unit with the same binary bit width as the data element.

[0099] Clause A8. The data processing apparatus according to Clause A3, wherein the total number of times the control unit issues the vector filtering instructions is equal to the sum of the binary bit width of the data elements in the first sort and the number of binary bits required for the batch identifier in the second sort.

[0100] Clause A9. The data processing apparatus according to Clause A1, wherein the arithmetic unit is further configured to: divide the second sorting result according to the original data size of each batch, and output the divided second sorting result to an off-chip storage unit.

[0101] Clause A10, a chip including a data processing device according to any one of Clauses A1-A9.

[0102] Clause A11, a board including the chip described in Clause A10.

[0103] Clause A12. A data processing method, comprising: concatenating multiple batches of data to be sorted into a merged data block, and generating a batch identifier corresponding to each data element in the merged data block; performing a first sorting on the merged data block associated with the batch identifier according to the values ​​of the data elements in the merged data block, to obtain a first sorting result; performing a second sorting on the first sorting result according to the batch identifier, wherein the second sorting is a stable sorting, to obtain a second sorting result, wherein data elements belonging to the same batch in the second sorting result are arranged consecutively and are internally ordered.

[0104] Clause A13, the data processing method according to Clause A12, wherein both the first sort and the second sort are least significant digit-first radix sorts.

[0105] Clause A14, the data processing method according to Clause A12, further includes: when performing the first sorting, invoking a vector filtering instruction to sort and distribute the data elements bit by bit according to the binary bit width of the data elements; when performing the second sorting, invoking a vector filtering instruction to sort and distribute the data elements bit by bit according to the binary bit width required for the batch identifier.

[0106] Clause A15, the data processing method according to Clause A14, wherein the number of binary bits required for the batch identifier is ceil(log2N), where N is the number of batches concatenated to the merged data block.

[0107] Clause A16, the data processing method according to Clause A14 or A15, wherein the batch identifier is represented using the same binary bit width as the data element.

[0108] Clause A17. The data processing method according to Clause A14, wherein the total number of times the vector filtering instruction is invoked is equal to the sum of the binary bit width of the data elements in the first sort and the number of binary bits required for the batch identifier in the second sort.

[0109] Clause A18. The data processing method according to Clause A12 further includes: dividing the second sorting result according to the original data size of each batch to obtain independent sorting results for each batch.

[0110] While numerous embodiments of this application have been shown and described herein, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Many modifications, alterations, and alternatives will arise for those skilled in the art without departing from the spirit and intent of this application. It should be understood that various alternatives to the embodiments of this application described herein may be employed in the practice of this application. The appended claims are intended to define the scope of protection of this application and therefore cover equivalents or alternatives within the scope of these claims.

Claims

1. A data processing apparatus, comprising a storage unit and a processing unit; wherein, The storage unit is configured to store multiple batches of data to be sorted; The arithmetic unit is configured as follows: Multiple batches of unsorted data are concatenated into a merged data block, and a batch identifier corresponding to each data element in the merged data block is generated. Based on the values ​​of data elements in the merged data block, perform a first sorting on the merged data block associated with the batch identifier to obtain a first sorting result; Based on the batch identifier, a second sort is performed on the first sorting result, and the second sort is a stable sort, to obtain a second sorting result, wherein the data elements belonging to the same batch in the second sorting result are arranged continuously and are internally ordered.

2. The data processing apparatus according to claim 1, wherein the first sorting and the second sorting are both radix sorting with least significant digit priority.

3. The data processing apparatus according to claim 1 or 2, further comprising: The control unit is configured to issue vector filtering instructions to the computing unit to control the computing unit to execute the first sorting and the second sorting sequentially.

4. The data processing apparatus according to claim 3, wherein the control unit is further configured to: Based on the available capacity of the storage unit, the number of batches of data to be sorted and concatenated into the merged data block is determined, wherein... The sum of the storage space occupied by the merged data block and the batch identifier does not exceed the available capacity of the storage unit.

5. The data processing apparatus according to claim 1, wherein the arithmetic unit is further configured as: When performing the first sorting, the data elements are sorted and distributed bit by bit according to their binary bit width; When performing the second sorting, the batch is sorted and distributed bit by bit according to the number of binary bits required by the batch identifier.

6. The data processing apparatus of claim 5, wherein the number of binary bits required for the batch identifier is ceil(log2N), where N is the number of batches concatenated to the merged data block.

7. The data processing apparatus according to claim 5 or 6, wherein the batch identifier is stored in the storage unit with the same binary bit width as the data element.

8. The data processing apparatus according to claim 3, wherein the total number of times the control unit issues the vector filtering instructions is equal to the sum of the binary bit width of the data elements in the first sort and the number of binary bits required for the batch identifier in the second sort.

9. The data processing apparatus according to claim 1, wherein the arithmetic unit is further configured as: Based on the original data size of each batch, the second sorting result is divided, and the divided second sorting result is output to the off-chip storage unit.

10. A chip comprising a data processing apparatus according to any one of claims 1-9.

11. A board comprising the chip as described in claim 10.

12. A data processing method, comprising: Multiple batches of unsorted data are concatenated into a merged data block, and a batch identifier corresponding to each data element in the merged data block is generated. Based on the values ​​of data elements in the merged data block, perform a first sorting on the merged data block associated with the batch identifier to obtain a first sorting result; Based on the batch identifier, a second sort is performed on the first sorting result, and the second sort is a stable sort, to obtain a second sorting result, wherein the data elements belonging to the same batch in the second sorting result are arranged continuously and are internally ordered.

13. The data processing method according to claim 12, wherein the first sorting and the second sorting are both radix sorting with least significant digit priority.

14. The data processing method according to claim 12, further comprising: When performing the first sorting, a vector filtering instruction is invoked to sort and distribute the data elements bit by bit according to their binary bit width; When performing the second sorting, a vector filtering instruction is invoked to sort and distribute the batches bit by bit according to the number of binary bits required by the batch identifier.

15. The data processing method of claim 14, wherein the number of binary bits required for the batch identifier is ceil(log2N), where N is the number of batches concatenated to the merged data block.

16. The data processing method according to claim 14 or 15, wherein the batch identifier is represented using the same binary bit width as the data element.

17. The data processing method according to claim 14, wherein the total number of times the vector filtering instruction is invoked is equal to the sum of the binary bit width of the data elements in the first sort and the number of binary bits required for the batch identifier in the second sort.

18. The data processing method according to claim 12, further comprising: Based on the original data size of each batch, the second sorting result is divided to obtain independent sorting results for each batch.

Images