Data processing device, data processing method, chip and electronic equipment

Abstract

This application discloses a data processing apparatus, a data processing method, a chip, and an electronic device, relating to the field of data processing technology. The data processing apparatus includes: a storage module configured to store x sets of original data vectors, each original data vector containing m data elements, where x and m are positive integers; and a computation module including a single instruction multiple data (SIMD) execution unit and a vector register group, the SIMD execution unit being coupled to the vector register group, and the vector register group being coupled to the storage module. The computation module is configured to: determine whether x is an integer multiple of the SIMD processing capability, split x, and obtain the top-k data elements in the original data vectors based on different data strategies. The algorithm of this application is flexible, requires no dedicated hardware sorting network, avoids dependence on dedicated hardware, and improves programming flexibility and versatility, facilitating the expansion and migration of large models across different hardware platforms.

Description

Technical Field

[0001] This application relates to the field of data processing technology, specifically to a data processing device, a data processing method, a chip, and an electronic device. Background Technology

[0002] In the technical framework of Large Language Models (LLM), the top-k operator is a core component balancing model efficiency and performance. Its core value lies in helping the model achieve efficient decision-making in complex natural language processing tasks by dynamically filtering key information. LLMs, represented by GPT, LLaMA, and BERT, need to complete core tasks such as text generation, semantic understanding, and logical reasoning. These tasks require the model to quickly locate key content from massive amounts of candidate information (such as words and knowledge fragments), and the top-k operator's characteristic of "taking the top k optimal solutions" precisely matches this high-frequency requirement.

[0003] Especially in Hybrid Expert Model (MoE) architectures, the top-k operator, as the core hub for achieving sparse activation and efficient computation, directly determines the model's resource allocation logic. Large MoE models refer to models that are split into multiple expert networks, using gating networks to dynamically select activated expert combinations for each input sample. The top-k operator is the "core engine" of this sparse activation technique, efficiently determining the subset of experts participating in the current computation by ranking expert matching scores, thus fundamentally breaking through the computational bottleneck of traditional dense models (such as Transformer). Because each layer of a traditional dense large model requires computation of all parameters of that layer, the computational load increases exponentially with model size (for example, the 175-parameter GPT-3 faces high inference costs). In contrast, the MoE large model achieves dynamic sparse activation through top-k routing and selects a small number of activation experts to participate in the computation, thereby significantly reducing the computational cost of the model. Therefore, while the model parameter scale expands to hundreds of billions or even trillions, the actual computational cost of the model can be controlled at a level comparable to that of small and medium-sized dense large models, thus significantly improving the scalability and economy of the model.

[0004] Given the crucial role of the top-k operator in large language models, researching and optimizing the computational efficiency of the top-k operator on hardware devices such as GPUs (Graphics Processing Units) / TPUs (Tensor Processing Units) (e.g., quickly extracting the indices and scores of the top k experts) will directly affect the training and inference speed of the MoE large model and become a key technical point connecting algorithm design and engineering implementation.

[0005] Existing technical solutions typically utilize dedicated hardware sorting networks to sort data in parallel, and then select the top-k data from the sorted results. While this approach, which relies on dedicated hardware, offers high execution efficiency, it requires software compilers to be customized to suit the characteristics of the dedicated hardware. This results in poor programming flexibility and versatility, and is not conducive to the expansion and migration of large models across different hardware platforms.

[0006] This section is intended to provide background or context for the embodiments of this application set forth in the claims. The description herein is not an admission that it is prior art simply because it is included in this section. Summary of the Invention

[0007] In order to solve at least one of the above-mentioned problems in the prior art, embodiments of this application provide a data processing apparatus, a data processing method, a chip, and an electronic device.

[0008] According to a first aspect of this application, a data processing apparatus is provided, comprising: a storage module configured to store x sets of original data vectors, the original data vectors containing m data elements, where x and m are positive integers; and a calculation module including a single instruction multiple data execution unit (SIMD) and a vector register group, the SIMD execution unit being coupled to the vector register group, the vector register group being coupled to the storage module, the calculation module being configured to: determine whether x is an integer multiple of the SIMD processing capability; in response to x being an integer multiple of the SIMD processing capability, load the x sets of original data vectors into the vector register group based on a first data processing strategy, and perform odd-even merge sort and merge sort using the SIMD execution unit to obtain the top-k data elements in the x sets of original data vectors, where k is a positive integer; in response to x not being an integer multiple of the SIMD processing capability, split x into x1 and x2, where x1 is an integer multiple of the SIMD processing capability and x2 is the remainder, and perform the split based on the first data processing strategy. The process involves: loading the x1 group of original data vectors into the vector register group; performing odd-even merge sort and merge sort using the single instruction multiple data (SIMD) unit to obtain the top-k data elements in the x1 group of original data vectors; determining whether x2 can be split into x21 and x22, where x21×n is an integer multiple of the SIMD processing capability, n is the number of subgroups obtained after dividing m data elements according to b granularity, and x22 is the remaining part; if x21 is not 0, then loading the x21 group of original data vectors into the vector register group based on the second data processing strategy; performing odd-even merge sort and merge sort using the SIMD unit to obtain the top-k data elements in the x21 group of original data vectors; if x22 is not 0, then loading the x22 group of original data vectors into the vector register group based on the third data processing strategy; performing odd-even merge sort and merge sort using the SIMD unit to obtain the top-k data elements in the x22 group of original data vectors.

[0009] According to a second aspect of this application, a data processing method is also provided, the method comprising: accessing x sets of original data vectors stored in a storage module, wherein the original data vectors contain m data elements, where m is a positive integer; determining whether x is an integer multiple of SIMD processing capability; in response to x being an integer multiple of SIMD processing capability, loading the x sets of original data vectors into the vector register group based on a first data processing strategy, and performing odd-even merge sort and merge sort using the single instruction multiple data execution unit to obtain the top-k data elements in the x sets of original data vectors, where k is a positive integer; in response to x not being an integer multiple of SIMD processing capability, splitting x into x1 and x2, where x1 is an integer multiple of SIMD processing capability and x2 is the remainder, and loading the x1 set of original data vectors into the vector register group based on the first data processing strategy, and performing odd-even merge sort and merge sort using the single instruction multiple data execution unit. Perform odd-even merge sort and merge sort to obtain the top-k data elements in the original data vector x1. Determine whether x2 can be split into x21 and x22, where x21×n is an integer multiple of the SIMD processing capability, n is the number of subgroups obtained after splitting m data elements according to b granularity, and x22 is the remaining part. In response to x21 not being 0, load the original data vector x21 into the vector register group based on the second data processing strategy, and perform odd-even merge sort and merge sort using the single instruction multiple data execution unit to obtain the top-k data elements in the original data vector x21. In response to x22 not being 0, load the original data vector x22 into the vector register group based on the third data processing strategy, and perform odd-even merge sort and merge sort using the single instruction multiple data execution unit to obtain the top-k data elements in the original data vector x22.

[0010] According to a third aspect of this application, embodiments of this application also provide a chip, the chip including the data processing apparatus described in any of the above embodiments.

[0011] According to a fourth aspect of this application, embodiments of this application also provide an electronic device, the electronic device comprising the chip described in any of the foregoing embodiments.

[0012] The data processing apparatus, data processing method, chip, and electronic device proposed in this application can select different processing methods to perform odd-even merge sort and merge sort according to the number of input original data vectors and the processing capability of SIMD, thereby obtaining the corresponding number of top-k data elements. The algorithm of this application is flexible, does not require a dedicated hardware sorting network, avoids dependence on dedicated hardware, and improves programming flexibility and versatility, which is conducive to the expansion and migration of large models on different hardware platforms. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings:

[0014] Figure 1 This is a schematic diagram of the structure of a data processing device provided in an embodiment of this application;

[0015] Figure 2 This is a flowchart illustrating the process of obtaining the top-k data elements in the original data vector based on a first data processing strategy, as provided in an embodiment of this application.

[0016] Figure 3 It is a matrix diagram composed of x groups of m elements of the original data vectors provided in the embodiments of this application;

[0017] Figure 4 It is a matrix diagram composed of the original data vectors of b elements in x21n groups provided in the embodiments of this application;

[0018] Figure 5 This is a schematic diagram of an 8-element parity merge sorting network provided in an embodiment of this application;

[0019] Figure 6 This is a schematic diagram of the first sub-vector matrix with an 8x8 grid provided in an embodiment of this application;

[0020] Figure 7 Is with Figure 6 A schematic diagram of the SIMD operations required for intra-block parity sorting;

[0021] Figure 8 This is a schematic diagram of the operation combination provided in the embodiments of this application;

[0022] Figure 9 This is a schematic diagram of merging and sorting two sets of 8-element data provided in an embodiment of this application;

[0023] Figure 10 This is a schematic diagram of the algorithm flow for selecting the largest or smallest k data points from m data points, provided in an embodiment of this application.

[0024] Figure 11 This is a comparison diagram of the data bit width and data precision processed by a single instruction in a vector processor using a SIMD architecture, provided in an embodiment of this application.

[0025] Figure 12This is a schematic diagram illustrating how operands in a parallel block of an odd-even merge sort network are mapped to a vector register, as provided in an embodiment of this application.

[0026] Figure 13 This is a comparative diagram of a trimmed merge sort method provided in an embodiment of this application and a conventional merge sort method;

[0027] Figure 14 This is a schematic diagram of the structure of a data processing apparatus provided in another embodiment of this application;

[0028] Figure 15 This is a schematic diagram of four 8-row, 8-column first sub-vector matrices provided in an embodiment of this application;

[0029] Figure 16 Is with Figure 15 A schematic diagram of the SIMD operations required for intra-block parity sorting;

[0030] Figure 17 This is a flowchart illustrating the execution of SIMD operations provided in the embodiments of this application;

[0031] Figure 18 This is a schematic diagram of two 8-row, 16-column element matrices provided in the embodiments of this application and the corresponding SIMD operations required for the first round of merge sorting;

[0032] Figure 19 Is with Figure 18 A diagram illustrating the SIMD operations required for the second round of merge sort;

[0033] Figure 20 This is a schematic diagram of an 8x8 first subvector matrix provided in an embodiment of this application;

[0034] Figure 21 This is a schematic diagram of the SIMD operation required for intra-block parity merge sorting of an 8x8 first subvector matrix provided in an embodiment of this application;

[0035] Figure 22 This is a schematic diagram of an 8-row, 16-column element matrix and the corresponding SIMD operations required for the first round of merge sort, provided in an embodiment of this application.

[0036] Figure 23 Is with Figure 22 A diagram illustrating the SIMD operations required for the second round of merge sort;

[0037] Figure 24 This is a schematic diagram comparing the bit width and data precision of SIMD processing according to an embodiment of this application;

[0038] Figure 25This is a schematic diagram illustrating how a comparison-swapping operation of four ordered sequences is mapped to ten SIMD operations, as provided in an embodiment of this application.

[0039] Figure 26 This is a schematic diagram illustrating how the comparison-swapping operations of four ordered sequences can be integrated into five SIMD operations in one embodiment of this application;

[0040] Figure 27 This is a schematic diagram illustrating how the comparison-swapping operations of two sets of ordered 8-element sequences can be integrated into 4 SIMD operations in one embodiment of this application;

[0041] Figure 28 This is a flowchart illustrating a data processing method provided in an embodiment of this application;

[0042] Figure 29 This is a block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0043] Reference will now be made in detail to exemplary embodiments of the invention, examples of which are illustrated in the accompanying drawings. Wherever possible, the same component reference numerals are used in the drawings and description to denote the same or similar parts.

[0044] The term "coupled (or connected)" as used throughout this specification (including the claims) may refer to any direct or indirect means of connection. For example, if the text describes a first device coupled (or connected) to a second device, it should be interpreted as the first device being directly connected to the second device, or the first device being indirectly connected to the second device through other devices or some means of connection. The terms "first," "second," etc., used throughout this specification (including the claims) are used to name components and are not intended to limit the upper or lower limit of the number of components, nor to limit the order of components. Furthermore, wherever possible, components / components / steps using the same reference numerals in the drawings and embodiments represent the same or similar parts. Components / components / steps using the same reference numerals or the same terms in different embodiments may be referred to mutually in the relevant descriptions.

[0045] Figure 1 This is a schematic diagram of the structure of a data processing device provided in an embodiment of this application, as shown below. Figure 1 As shown, the data processing apparatus includes:

[0046] Storage module 1 is configured to store x sets of original data vectors, wherein the original data vectors contain m data elements, where x and m are positive integers.

[0047] Computation module 2 includes a single instruction multiple data (SIMD) execution unit 21 and a vector register set 22, wherein the SIMD execution unit 21 is coupled to the vector register set 22, and the vector register set 22 is coupled to the storage module 1. This computation module is configured to: determine whether x is an integer multiple of the SIMD processing capability; and respond to x being a SIMD (Single Instruction Multiple Data) processing capability. If x is an integer multiple of the SIMD processing capability, then based on the first data processing strategy, the x groups of original data vectors are loaded into the vector register group. The single-instruction multiple-data (SIMD) unit is used to perform odd-even merge sort and merge sort to obtain the top-k data elements in the x groups of original data vectors, where k is a positive integer. If x is not an integer multiple of the SIMD processing capability, then x is split into x1 and x2, where x1 is an integer multiple of the SIMD processing capability and x2 is the remainder. Based on the first data processing strategy, the x1 group of original data vectors is loaded into the vector register group. The single-instruction multiple-data (SIMD) unit is used to perform odd-even merge sort and merge sort to obtain the top-k data elements in the x1 group of original data vectors. It is then determined whether x2 can be split into... x21 and x22, where x21×n is an integer multiple of the SIMD processing capability, n is the number of subgroups obtained after dividing m data elements according to b granularity, and x22 is the remaining part; in response to x21 not being 0, the original data vector of group x21 is loaded into the vector register group based on the second data processing strategy, and parity merge sort and merge sort are performed using the single instruction multiple data execution unit to obtain the top-k data elements in the original data vector of group x21; in response to x22 not being 0, the original data vector of group x22 is loaded into the vector register group based on the third data processing strategy, and parity merge sort and merge sort are performed using the single instruction multiple data execution unit to obtain the top-k data elements in the original data vector of group x22.

[0048] The data processing apparatus proposed in this application can select different processing methods to perform odd-even merge sort and merge sort according to the number of input original data vectors and the processing capability of SIMD, thereby obtaining the corresponding number of top-k data elements. The algorithm of this application is flexible, does not require a dedicated hardware sorting network, avoids dependence on dedicated hardware, and improves programming flexibility and versatility, which is conducive to the expansion and migration of large models on different hardware platforms.

[0049] In some embodiments of this application, such as Figure 2 As shown, the above-mentioned calculation module loads the x groups or x1 groups of original data vectors into the vector register group based on the first data processing strategy, and uses the single instruction multiple data execution unit to perform odd-even merge sort and merge sort to obtain the top-k data elements in the x groups or x1 groups of original data vectors, including:

[0050] Step S201: Divide the original data vectors of group x or group x1 into a first sub-vector matrix with y rows and b columns, where y is the data processing capability of a single instruction multiple data execution unit, and the insufficient parts in the rows and columns of the first sub-vector matrix are supplemented according to the top-k sorting requirements.

[0051] Since both x and x1 are integer multiples of the SIMD processing capability, i.e., integer multiples of y, dividing the original data vectors of x groups or x1 groups into the first sub-vector matrix of y rows and b columns will not affect the utilization rate of SIMD. For example, when the SIMD data processing bit width is 128 bits and the data precision is 16 bits, the number of data processed by a single SIMD instruction is 8. Therefore, when x is an integer multiple of 8, its SIMD utilization rate can reach 100%. If x cannot meet this requirement (e.g., x=12), then the SIMD utilization rate is only... This situation can be seen in [reference needed]. Figure 3 It is a matrix composed of m elements of the original data vectors in x groups. Since x is an integer multiple of y, the data can be mapped to a vector register with a bit width equal to that of SIMD data processing according to the x dimension, which facilitates subsequent vector processing.

[0052] In this step, assuming the SIMD processing width is 128 bits and the data precision is 16 bits, the SIMD execution unit can process 8 data items in parallel with a single instruction, i.e., y is 8. If x or x1 is 8 at this time, and each batch of data has 32 data elements, then it is necessary to select the top-8 data elements from each of these 8 batches of data. b is an integer greater than or equal to 8 and a power of 2, i.e., b is also 8 (2^2 + 1 / 2). 3 Therefore, the data from these 8 batches can be divided into 4 sub-vector matrices, each with 8 rows and 8 columns.

[0053] If each of the above 8 batches contains 31 data elements, then during the partitioning, the last first sub-vector matrix will only have 7 columns. In this case, the last column of data elements can be supplemented according to the top-k sorting requirements. For example, when the top-k sorting takes the largest k elements, a column of sufficiently small numbers, such as infinitesimally small numbers, can be added; when the top-k sorting takes the smallest k elements, a column of sufficiently large numbers, such as infinitesimally large numbers, can be added. In this case, the supplemented data elements will not affect the final sorting result, and therefore will not affect the final top-k value selection.

[0054] Step S202: Load the first sub-vector matrix column by column into the vector register group, and use the single instruction multiple data execution unit to perform parity merge sorting to achieve in-row sorting of the first sub-vector matrix.

[0055] Since the number of rows in the first subvector matrix is ​​y, which is exactly the processing capability of SIMD, and the data in the same column of the first subvector matrix are independent of each other and do not affect each other, the first subvector matrix can be loaded into the vector register group column by column, and the two columns of data can be compared and exchanged simultaneously using the single instruction multiple data execution unit.

[0056] Step S203: Load the first sub-vector matrix after row sorting into the vector register group column by column, and use the single instruction multiple data execution unit to perform one or more rounds of merge sorting to obtain the top-k data elements in the x groups or x1 groups of original data vectors.

[0057] In some embodiments of this application, the above-mentioned calculation module loads x21 sets of original data vectors into the vector register group based on the second data processing strategy, and uses the single instruction multiple data execution unit to perform odd-even merge sort and merge sort to obtain the top-k data elements in the x21 sets of original data vectors, including:

[0058] Step S401: Divide the m data elements of each group in x21 into n subgroups with granularity b, to obtain a subvector matrix of x21n rows and b columns.

[0059] As mentioned above, x21 is a subset of x2. When the amount of data is large, a large amount of data has already been processed in x1, and the amount of data in x2 is not very large.

[0060] Since the number of data points *m* in a vector dataset is usually a multiple of *b*, when using the odd-even merge sort algorithm, the vector dataset with *m* elements can be split into *n* sub-vectors with a granularity of *b*. In this case, each sub-vector is independent and can be executed in parallel. For example... Figure 4 As shown, dividing the m data elements of each group in x21 into n subgroups with granularity b will result in a subvector matrix of x21n rows and b columns.

[0061] Step S402: Divide the sub-vector matrix into a first sub-vector matrix with y rows and b columns, where y is the number of data processing capabilities of a single instruction multiple data execution unit. The insufficient parts in the rows and columns of the first sub-vector matrix are supplemented according to the top-k sorting requirements.

[0062] Since x21n is an integer multiple of the SIMD processing capability y, the aforementioned x21n row b column subvector matrix can be further divided into a y row b column first subvector matrix. Then, elements with the same index in these first subvector matrices are mapped to a vector register equal to the SIMD data processing bit width for subsequent vector processing. This scheme can achieve 100% utilization of SIMD. Furthermore, the method for supplementing insufficient rows and columns in the first subvector matrix in this step is the same as in step S201 above.

[0063] Step S403: Load the first sub-vector matrix column by column into the vector register group, and use the single instruction multiple data execution unit to perform parity merge sorting to achieve in-row sorting of the first sub-vector matrix.

[0064] Step S404: Load the first sub-vector matrix after row sorting into the vector register group column by column, and use the single instruction multiple data execution unit to perform one or more rounds of merge sorting to obtain the top-k data elements in the x21 original data vectors.

[0065] In some embodiments of this application, the above-mentioned calculation module loads the x22 sets of original data vectors into the vector register group based on a third data processing strategy, and uses the single instruction multiple data execution unit to perform odd-even merge sort and merge sort to obtain the top-k data elements in the x22 sets of original data vectors, including:

[0066] Perform the following operation on each of the original data vectors in group x22:

[0067] Step S601: Split the original data vector based on the target k value to obtain multiple first data vectors.

[0068] Step S602: Map the data elements in each of the first data vectors to the vector register group respectively, and use the single instruction multiple data execution unit to perform parallel comparison-exchange operations on the data elements in the vector register group, so as to realize the parity merge sort of the data elements in each of the first data vectors respectively, and obtain multiple first ordered data vectors.

[0069] Step S603: Merge and sort the data elements in the plurality of first ordered data vectors to obtain the top-k data elements in the original data vector.

[0070] As mentioned above, after splitting the x22 group by granularity b, none of the subgroups can satisfy the requirement that they are integer multiples of the SIMD processing capacity. Therefore, x22 only represents a small amount of data, such as only one or two sets of original data vectors. Therefore, the steps shown in steps S601 to S603 can be performed on each set of original data vectors to make full use of the SIMD processing capacity.

[0071] In some embodiments of this application, in steps S202 and S403 above, the calculation module loads the first sub-vector matrix column-wise into the vector register group, and uses the single instruction multiple data execution unit to perform parity merge sorting to achieve row-wise sorting of the first sub-vector matrix. Further, this may include: mapping the data elements in the first sub-vector matrix column-wise to the vector register group, using the single instruction multiple data execution unit to operate on the data elements in the vector register group, and performing block-wise parity merge sorting of the first sub-vector matrix to obtain a first sub-vector matrix arranged in row-wise order within the block.

[0072] Odd-even merge sort is a parallel sorting algorithm based on a comparison network. Its core idea is to recursively merge elements at odd and even positions to construct a data-independent sorting network. This algorithm has significant advantages in hardware implementation, especially suitable for applications requiring high parallelism. Figure 5 The diagram shows a parity merge sort network with 8 elements. Wherein:

[0073] 0~7 represents data elements 0~7, with each line corresponding to one data element.

[0074] The red dashed box (parallelizable block) represents the independent operation unit divided by the dashed box. The operations within each block can be executed simultaneously, demonstrating the parallelism of the algorithm.

[0075] Gray arrows: Indicate comparison-swap operations. The direction of the arrow reflects the direction of data comparison. For example, up and down arrows indicate comparison of adjacent or specific data positions; if the order is incorrect, the data is swapped.

[0076] In odd-even merge sort, the sorting process is divided into multiple parallel blocks. The comparison-swap operations within each block can be executed simultaneously, which facilitates the parallel use of hardware resources. Different parallel blocks need to be executed from left to right. The data sorting is completed after all the operations in the parallel blocks have been executed.

[0077] Based on this feature of the parity merge sort network, in some embodiments of this application, the above-mentioned mapping of data elements in the first sub-vector matrix to the vector register group by column, and the use of a single instruction multiple data execution unit to operate on the data elements in the vector register group to perform intra-block parity merge sort on the first sub-vector matrix, resulting in an intra-block row-ordered first sub-vector matrix, may further include:

[0078] Step S801: Map the data elements in the first sub-vector matrix column by column to the vector register group according to the first mapping rule. Use the single instruction multiple data execution unit to perform parallel comparison-exchange operations on the data elements at the same position in the vector register group to obtain two first sub-data vectors. Then, reverse map the data elements in the two first sub-data vectors back to the address space of the storage module storing the first sub-vector matrix according to the first mapping rule, so that the first sub-vector matrix stored in the storage module is updated to the second sub-vector matrix.

[0079] Since the first subvector matrix is ​​a matrix with y rows and b columns, the number of data in one column is exactly the amount of data (i.e., y) that a single instruction of the SIMD execution unit can process in parallel. In this embodiment, each column of data in the first subvector matrix can be grouped together, which is equivalent to... Figure 5 A data element from 0 to 7. For example... Figure 6 As shown, taking an 8x8 matrix as an example, the first subvector matrix can be divided into eight groups, from group 0 to group 7, which correspond to... Figure 5 The first subvector matrix contains eight data elements, 0-7. Correspondingly, the intra-block sorting of this first subvector matrix can include, for example... Figure 7 The 19 SIMD operations shown can compare and swap two sets of data simultaneously. For example, the 8 data in group 0 can be compared and swapped with the 8 corresponding data in group 1.

[0080] As described above, this embodiment can perform comparison-swap operations on 8 pairs of data elements simultaneously in a single SIMD operation. The complete execution steps of a single comparison-swap operation on a pair of data elements are described below. The execution steps of a single comparison-swap operation based on the RISC-V vector extension instruction architecture are as follows:

[0081] Step 1 (Data Reading): Read the two sets of vectors vs1 and vs2 to be compared from the storage module (e.g., SRAM, static random access memory), as well as the index vectors vs3 and vs4 that record the positions of the data elements in the vectors, and store vs1, vs2, vs3 and vs4 in the vector register to prepare data for subsequent operations.

[0082] In step 1, data elements in the first data vector stored in the storage module can be read based on the above data mapping rules. The required data elements are extracted and formed into two sets of vectors vs1 and vs2, which are then stored in two vector registers respectively.

[0083] Step 2 (Element Comparison): Use the compare instruction in the RISC-V vector extension instruction set, such as vmfne (to determine if they are equal), vmflt (to determine if they are less than), vmfle (to determine if they are less than or equal to), vmfgt (to determine if they are greater than), and vmfge (to determine if they are greater than or equal to), to compare the elements at the same position in vectors vs1 and vs2, and obtain a bit-mask value (represented by v0) that indicates the size relationship between the elements in the two sets of vectors. This value is used to identify the comparison result of each pair of elements (such as greater than, less than, or equal to).

[0084] Step 3 (Selection and Merging): Using the merge instruction in the vector extension instruction set, such as vmerge, vectors vs1 and vs2 are selected and merged based on the bit-mask value v0, generating two new vectors vd1 and vd2; simultaneously, vectors vs3 and vs4 are selected and merged based on v0, resulting in two new vectors vd3 and vd4. This step achieves the selection and recombination of vector element values ​​and indices based on the comparison results.

[0085]

[0086]

[0087]

[0088]

[0089] Step 4 (Result Write-back): Assign the new vectors vd1 and vd2 to vs1 and vs2 respectively, or write them back to the storage module directly; similarly, assign the new vectors vd3 and vd4 to vs3 and vs4 respectively, or write them back to the storage module, thereby completing a comparison-exchange operation on the data elements in the first data vector.

[0090] Similar to a single-pair data element comparison-swapping operation, this embodiment maps one column of data elements into one vector register group and simultaneously maps another column of data elements to be compared into another vector register group. Then, the comparison-swapping operation is performed simultaneously according to the above process. In this embodiment, after each comparison-swapping operation of two columns of elements is completed, the result is immediately reverse-mapped back to the address space of the storage module storing the first sub-vector matrix according to the first mapping rule, so that the first sub-vector matrix stored in the storage module is updated to the second sub-vector matrix.

[0091] Step S802: Map the data elements in the second sub-vector matrix column by column to the vector register group according to the second mapping rule. Use the single instruction multiple data execution unit to perform parallel comparison-swapping operations on the data elements at the same position in the vector register group to obtain two second sub-data vectors. Then, reverse map the data elements in the two second sub-data vectors back to the address space of the storage module storing the second sub-vector matrix according to the second mapping rule, so that the second sub-vector matrix stored in the storage module is updated to the third sub-vector matrix.

[0092] The execution process of step S802 is similar to that of step S801, the difference being that in step S802, data elements are mapped and inverse mapped according to the second mapping rule. Figure 7 For example, step S801 handles the comparison-swap operation between group 0 and group 1, while this step handles the comparison-swap operation between group 2 and group 3. In other words, the difference between step S802 and step S801 lies only in the different data elements to be operated on, and therefore the corresponding data mapping rules are also different. Their execution processes are similar, so the execution process of step S802 will not be described in detail here. For details, please refer to the detailed description of the execution process of step S801 above.

[0093] Step S803: Continue in this manner until the comparison-exchange operation required for one round of parity merge sort is completed, that is, the parity merge sort within the block of the first sub-vector matrix is ​​completed, resulting in a first sub-vector matrix arranged in row order within the block.

[0094] After completing the comparison-swap operations required for one round of odd-even merge sort, for example Figure 7 After 19 SIMD operations, the parity merge sort of the first data vector within the block is completed, resulting in a first sub-vector matrix arranged in row order within the block.

[0095] In some embodiments of this application, the above-described mapping of data elements in the i-th sub-vector matrix to the vector register group column-by-column according to the i-th mapping rule includes:

[0096] Step S1201: Based on the odd-even merge sort algorithm, determine the pairs of data elements in the i-th sub-vector matrix that need to be compared and swapped column by column, where i=1,2.

[0097] Step S1202: Map the two data elements in each data element pair to the same position in two different vector registers, wherein the data elements in each data element pair are mapped to the two different vector registers.

[0098] This has already been described in the preceding description of step S801, and will not be repeated here.

[0099] In the embodiments corresponding to steps S801 to S803 above, after each SIMD operation, the output results (including the numerical vector and the position index vector) after the comparison-swap operation are written back to SRAM to release the occupied vector register. However, frequent SRAM reads and writes increase additional power consumption and latency. In fact, the output results of some SIMD operations may be the source operands of subsequent SIMD operations. Therefore, these output results can be kept in the vector register without being written back to SRAM, waiting for subsequent SIMD operations to process them. They can be written back to SRAM together after all these related operations are completed. Figure 8 For example, multiple SIMD operations in combinations 1 to 4 will reuse some of the same source operands (for example, all 5 SIMD operations in combination 1 will use data 0 to 3). Therefore, these SIMD operations in each combination can be combined together for execution, and then the final output can be written back to SRAM.

[0100] Based on the above reasons, in some embodiments of this application, the process of mapping data elements in the first sub-vector matrix to a vector register group by column, and using a single instruction multiple data execution unit to operate on the data elements in the vector register group to perform intra-block parity merge sorting on the first sub-vector matrix to obtain an intra-block row-ordered second sub-vector matrix may further include:

[0101] Step S141: Based on the number of vector register groups, identify one or more operation combinations consisting of multiple consecutive single instruction multiple data comparison-swap operations during the intra-block parity merge sorting process of the first sub-vector matrix, wherein the output data of a preceding comparison-swap operation in the operation combination is the input data of a subsequent comparison-swap operation.

[0102] In this step, identifying one or more operation combinations consisting of multiple sequentially related single-instruction multiple-data comparison-exchange operations during the intra-block parity merge sorting process of the first sub-vector matrix based on the number of vector register sets may further include: determining a starting parallel block based on the total number of available vector registers in the vector register set being a power of 2, where n is a positive integer and the starting parallel block is the nth parallel block in the intra-block parity merge sorting process; starting from the starting parallel block, searching towards its preceding parallel blocks to identify a set of consecutive single-instruction multiple-data comparison-exchange operations that have sequential relationships in data processing; and determining this set of consecutive single-instruction multiple-data comparison-exchange operations as an operation combination.

[0103] For example, when the total number of available vector registers in the vector register set is 8, n is 3. Figure 8 For example, it can be determined Figure 5 The third parallel block from left to right is the starting parallel block. Then, the search is performed on the preceding parallel blocks starting from this starting parallel block. For example, if the data element operated on in this parallel block is (1, 2), where 1 depends on the comparison result of (1, 3) in the second parallel block, and 2 depends on the comparison result of (0, 2) in the second parallel block; and 0, 1, 2, 3 in the second parallel block depend on the comparison results of (0, 1) and (2, 3) in the first parallel block, then the five operations (0, 1), (2, 3), (0, 2), (1, 3), and (1, 2) can be identified as an operation combination. When the compare-swap operation of (1,2) in the third parallel block is executed, its eight register groups can just store the results of the previous four compare-swap operations (0,1), (2,3), (0,2), and (1,3), but cannot continue to store the compare-swap result of (1,2). Therefore, after the compare-swap operation of (1,2) is completed, the result needs to be written back to SRAM.

[0104] As can be seen from the above, when the number of vector registers is large enough, it is possible to complete all the operations required for parity merge sorting before writing them all back to SRAM. However, the number of vector register groups is usually relatively small. Therefore, the method in this embodiment can reduce the number of writes back to SRAM based on the number of vector register groups, thereby reducing additional power consumption and latency.

[0105] Step S142: Load the source data required for the first compare-swap operation in the first operation combination from the first sub-vector matrix in the storage module into the vector register group. The source data is the column data in the first sub-vector matrix.

[0106] Still with Figure 8 Taking combination 1 as an example, the first compare-swap operation is (0, 1), which corresponds to Figure 6 The comparison operation between group 0 and group 1 is performed in the middle, so the data of group 0 and group 1 are loaded from the first sub-vector matrix in the storage module into the vector register group.

[0107] Step S143: Execute multiple single-instruction multiple-data compare-swap operations in the first operation combination in sequence, and keep the intermediate result data generated by each compare-swap operation in the vector register group for use in subsequent compare-swap operations.

[0108] The intermediate result data after the (group 0, group 1) compare-swap operation is kept in the vector register group for subsequent (group 0, group 2), (group 1, group 3), and (group 1, group 2) compare-swap operations.

[0109] Step S144: After all the comparison-swap operations in the first operation combination are completed, write the final output back to the storage module to update the first sub-vector matrix stored in the storage module, making it the second sub-vector matrix.

[0110] That is, after the comparison-swap operation in combination 1 (group 1, group 2) is completed, the final output result is written back to the storage module to update the first sub-vector matrix stored in the storage module.

[0111] Step S145: Load the source data required for the first compare-swap operation in the second operation combination from the second sub-vector matrix in the storage module into the vector register group. The source data is the column data in the second sub-vector matrix.

[0112] by Figure 8 Taking combination 2 as an example, the source data required for the first comparison operation (group 4, group 5) is loaded from the second sub-vector matrix in the storage module into the vector register group.

[0113] Step S146: Execute multiple single-instruction multiple-data compare-swap operations in the second operation combination in sequence, and keep the intermediate result data generated by each compare-swap operation in the vector register group for use in subsequent compare-swap operations.

[0114] The intermediate result data after the (group 4, group 5) compare-swap operation is kept in the vector register group for subsequent (group 4, group 6), (group 5, group 7) and (group 5, group 6) compare-swap operations.

[0115] Step S147: After all the comparison-swap operations in the second operation combination are completed, write the final output back to the storage module to update the second sub-vector matrix stored in the storage module again, so that it becomes the third sub-vector matrix.

[0116] That is, after the comparison-swap operation in combination 2 (group 5, group 6) is completed, the final output result is written back to the storage module to update the first subvector matrix stored in the storage module.

[0117] Step S148: Continue in this manner until the comparison-exchange operation required for one round of parity merge sort is completed, that is, the parity merge sort of the first sub-vector matrix within the block is completed, resulting in a first sub-vector matrix arranged in row order within the block.

[0118] In some embodiments of this application, the above-mentioned calculation module loads the first sub-vector matrix after row sorting into the vector register group column by column, and uses the single instruction multiple data execution unit to perform one or more rounds of merge sorting to obtain the top-k data elements in the x groups, x1 groups, or x21 groups of original data vectors, including:

[0119] Step S151: The first sub-vector matrix after in-row sorting is grouped into pairs by row and merged to obtain a fourth sub-vector matrix with c rows and 2b columns arranged in in-row sorting; the first b data elements of each row in the fourth sub-vector matrix are extracted to obtain a fifth sub-vector matrix with c rows and b columns arranged in in-row sorting.

[0120] Step S152: Group the fifth sub-vector matrix into pairs by row and merge and sort them to obtain a sixth sub-vector matrix with d rows and 2b columns arranged in order within the rows.

[0121] Step S153: Extract the first b data elements of each row in the sixth sub-vector matrix to obtain the seventh sub-vector matrix with d rows and b columns arranged in order within the rows.

[0122] Step S154: Continue in this manner until the merged result of x rows and 2b columns is obtained. Extract the first k columns of data elements from the merged result as the top-k data elements in the original data vector of group x, group x1, or group x21.

[0123] Taking the extraction of the top-k data elements from a set of elements as an example, such as... Figure 9 The diagram shows a merge sort of two sets of 8-element data. After the merge sort, a sorted vector of 16 elements is obtained, and then the top-k data elements are taken from this sorted vector.

[0124] In some embodiments of this application, the above-mentioned merging and sorting of the i-th subvector matrix by pairwise groups on a row-by-row basis includes:

[0125] Step S171: Map the data elements in the grouped i-th sub-vector matrix column by column to the vector register group according to the third mapping rule. Use the single instruction multiple data execution unit to perform parallel comparison-swapping operations on the data elements at the same position in the vector register group to obtain two first intermediate data vectors. Then, reverse map the data elements in the two first intermediate data vectors back to the address space of the storage module storing the i-th sub-vector matrix according to the first mapping rule, so that the first sub-vector matrix stored in the storage module is updated to the first intermediate vector matrix.

[0126] Similar to odd-even merge sort, when performing merge sort, the data elements in the i-th sub-vector matrix are mapped to the vector register group column by column, so that a single SIMD operation can complete the comparison and swap operation of two columns (two groups) of data.

[0127] Step S172: Map the data elements in the first intermediate vector matrix column by column to the vector register group according to the fourth mapping rule. Use the single instruction multiple data execution unit to perform parallel comparison-swapping operations on the data elements at the same position in the vector register group to obtain two second intermediate data vectors. Then, reverse map the data elements in the two second intermediate data vectors back to the address space of the storage module storing the first intermediate vector matrix according to the fourth mapping rule, so that the first intermediate vector matrix stored in the storage module is updated to the second intermediate vector matrix.

[0128] Step S173: Continue in this manner until the comparison-exchange operation required for one round of merge sort is completed, that is, the merge sort of the i-th sub-vector matrix is ​​completed, resulting in a vector matrix with 2b columns arranged in order within rows, where i=1,5.

[0129] In some embodiments of this application, mapping data elements in the grouped i-th sub-vector matrix to the vector register group column-by-column according to a third mapping rule includes: selecting two columns of data from the i-th sub-vector matrix according to the comparison and exchange rules of the merge sort algorithm, and mapping them to two different vector registers respectively. Mapping data elements in the first intermediate vector matrix column-by-column to the vector register group according to a fourth mapping rule includes: selecting two columns of data from the first intermediate vector matrix according to the comparison and exchange rules of the merge sort algorithm, and mapping them to two different vector registers respectively.

[0130] In some embodiments of this application, mapping data elements in the grouped i-th sub-vector matrix to the vector register group column-by-column according to a third mapping rule includes: selecting two columns of data from the i-th sub-vector matrix according to the comparison and exchange rules of the merge sort algorithm, and mapping them to two different vector registers respectively. Mapping data elements in the first intermediate vector matrix column-by-column to the vector register group according to a fourth mapping rule includes: selecting two columns of data from the first intermediate vector matrix according to the comparison and exchange rules of the merge sort algorithm, and mapping them to two different vector registers respectively.

[0131] In the embodiments corresponding to steps S171 to S173 above, after each SIMD operation, the output result (including the numerical vector and the position index vector) after the comparison-swap operation is written back to SRAM to release the vector register. Similar to merge sort, frequent SRAM reads and writes increase power consumption and latency. In fact, the output result of some SIMD operations may be the source operand for subsequent SIMD operations (see [link to documentation] for details). Figure 9As shown in the figure, these output results can be kept in the vector register without being written back to SRAM, and then written back to SRAM together after all the related operations are completed.

[0132] Based on the above reasons, in some embodiments of this application, the above-mentioned merging and sorting of the i-th subvector matrix by grouping it into pairs by row units may further include:

[0133] Step S181: Based on the number of vector register groups, identify one or more operation combinations consisting of multiple consecutive single-instruction multiple-data comparison-swap operations during the merge sorting process of grouping the i-th sub-vector matrix into pairs by row, wherein the output data of a preceding comparison-swap operation in the operation combination is the input data of a subsequent comparison-swap operation.

[0134] In this step, based on the number of vector register sets, one or more operation combinations consisting of multiple sequentially related single-instruction multiple-data comparison-exchange operations are identified during the merge sorting process of grouping the i-th sub-vector matrix into pairs by row. This can further include: determining a starting parallel block based on the total number of available vector registers in the vector register set being a power of 2, where n is a positive integer, and this starting parallel block is the nth parallel block in the merge sorting process; starting from this starting parallel block, searching towards its preceding parallel blocks to identify a set of consecutive single-instruction multiple-data comparison-exchange operations that have a sequential relationship in data processing; and determining this set of consecutive single-instruction multiple-data comparison-exchange operations as an operation combination.

[0135] For example, when the total number of available vector registers in the vector register set is 8, n is 3. Figure 9For example, we can determine the third parallel block from the left as the starting parallel block. Then, we start searching the preceding parallel blocks from this starting parallel block. For example, if the data element operated on in this parallel block is (2, 4), where 2 depends on the comparison result of (2, 10) in the first parallel block, and 4 depends on the comparison result of (4, 8) in the second parallel block; and 4 in the second parallel block depends on the comparison result of (4, 12) in the first parallel block, and 8 in the two parallel blocks depends on the comparison result of (0, 8) in the first parallel block. Therefore, the five operations (0, 8), (2, 10), (4, 12), (4, 8), and (2, 4) can be determined as an operation combination. When the compare-swap operation of (2,4) in the third parallel block is executed, its eight register groups can just store the results of the previous four compare-swap operations (0,8), (2,10), (4,12), and (4,8), but cannot continue to store the compare-swap result of (2,4). Therefore, after the compare-swap operation of (2,4) is completed, the result needs to be written back to SRAM.

[0136] As can be seen from the above, when the number of vector registers is large enough, it is possible to complete all the operations required for parity merge sorting before writing them all back to SRAM. However, the number of vector register groups is usually relatively small. Therefore, the method in this embodiment can reduce the number of writes back to SRAM based on the number of vector register groups, thereby reducing additional power consumption and latency.

[0137] Step S182: Load the source data required for the first compare-swap operation in the third operation combination from the i-th sub-vector matrix in the storage module into the vector register group, wherein the source data is the column data in the i-th sub-vector matrix.

[0138] The third operation combination and the subsequent fourth operation combination are used here to distinguish them from the corresponding concepts in the embodiments of steps S141 to S148, and specifically refer to the multiple operation combinations in this embodiment.

[0139] Taking the operation combination (0, 8), (2, 10), (4, 12), (4, 8) and (2, 4) in the aforementioned steps as an example, the first comparison-swap operation is (0, 8), which corresponds to the data of group 0 and group 8 after the two i-th sub-vector matrices are grouped and arranged. Therefore, the data of group 0 and group 8 are loaded from the two i-th sub-vector matrices in the storage module into the vector register group.

[0140] Step S183: Execute multiple single-instruction multiple-data compare-swap operations in the third operation combination in sequence, and keep the intermediate result data generated by each compare-swap operation in the vector register group for use in subsequent compare-swap operations.

[0141] For example, the intermediate result data after the (group 0, group 8) compare-swap operation is kept in the vector register group for subsequent (group 4, group 8) compare-swap operations.

[0142] Step S184: After all the comparison-swap operations in the third operation combination are completed, the final output result is written back to the storage module, so that the i-th sub-vector matrix stored in the storage module is updated to the third intermediate vector matrix.

[0143] That is, after the comparison-swap operation in combination 3 (group 2, group 4) is completed, the final output result is written back to the storage module to update the i-th sub-vector matrix stored in the storage module to the third intermediate vector matrix.

[0144] Step S185: Load the source data required for the first compare-swap operation in the fourth operation combination from the third intermediate vector matrix in the storage module into the vector register group, wherein the source data is the column data in the third intermediate vector matrix.

[0145] Still with Figure 9 For example, as shown in the figure, the five operations (3, 11), (5, 13), (1, 9), (5, 9), and (3, 5) can be combined as a fourth operation. The first comparison-swap operation is (3, 11), which corresponds to the data of group 3 and group 11 after the two i-th sub-vector matrices are grouped and arranged. Therefore, the data of group 3 and group 11 are loaded from the two i-th sub-vector matrices in the storage module into the vector register group.

[0146] Step S186: Execute multiple single-instruction multiple-data compare-swap operations in the fourth operation combination in sequence, and keep the intermediate result data generated by each compare-swap operation in the vector register group for use in subsequent compare-swap operations.

[0147] For example, the intermediate result data after the (group 3, group 11) compare-swap operation is kept in the vector register group for subsequent (group 3, group 5) compare-swap operations.

[0148] Step S187: After all the comparison-swap operations in the fourth operation combination are completed, the final output result is written back to the storage module, so that the third intermediate vector matrix stored in the storage module is updated to the fourth intermediate vector matrix.

[0149] That is, after the comparison-swap operation in combination 4 (group 2, group 4) is completed, the final output result is written back to the storage module to update the third intermediate vector matrix stored in the storage module to the fourth intermediate vector matrix.

[0150] Step S188: Continue in this manner until the comparison-exchange operations required for one round of merge sort are completed, that is, the merge sort of the i-th subvector matrix after pairwise grouping is completed, resulting in a vector matrix with 2b columns arranged in order within rows, where i=1,5.

[0151] In some embodiments of this application, the calculation module in step S601 splits the original data vector based on the target k value to obtain multiple first data vectors, including: calculating an integer greater than or equal to the power of 2 of the target k value; and splitting the original data vector using the integer greater than or equal to the power of 2 of the k value as the grouping granularity to obtain multiple first data vectors.

[0152] For example, Figure 10 This demonstrates the process of selecting the k largest or smallest data points from m data points. The specific steps are as follows:

[0153] The first step is to determine the grouping granularity: Based on the value of k, determine an integer b that is a power of 2 greater than or equal to k. For example, if k=5, then b=8 (2 to the power of 3).

[0154] The second step is data grouping: the m data points are divided into m / b groups with a grouping granularity of b.

[0155] The third step is to sort the data within each group: use the odd-even merge sort algorithm to sort the data within each group, resulting in m / b ordered groups of data.

[0156] The fourth step is the merge sort loop: group the multiple sets of ordered data into pairs and perform merge sort; then extract the first b data from each merge sorted data set to obtain a new set of ordered data; check if the number of ordered data sets is equal to 1. If it is not equal to 1, repeat the merge sort steps.

[0157] Step 5, extract the results: When the number of ordered data groups is equal to 1, extract the first k data from the b ordered data in that group, which are the k largest or smallest data selected in the end.

[0158] In some embodiments, the calculation module 2 maps a data element from one of the first data vectors to the vector register group 22 each time, and uses the single instruction multiple data execution unit 21 to perform parallel comparison-swapping operations on the data elements in the vector register group 22, thereby performing parity merge sort on the data elements in the first data vector to obtain a first ordered data vector; or, the calculation module 2 maps at least two data elements from the first data vectors to the vector register group 22 each time, and uses the single instruction multiple data execution unit 21 to perform parallel comparison-swapping operations on the data elements in the vector register group 22, thereby performing parallel parity merge sort on the data elements in the at least two first data vectors to obtain a first ordered data vector corresponding to each of the at least two first data vectors.

[0159] Specifically, the calculation module 2 can sequentially perform parity merge sort on the data elements in each of the first data vectors, or it can perform parity merge sort on the data elements in multiple first data vectors in parallel. The number of first data vectors processed in parallel each time can be determined according to the SIMD (Single Instruction Multiple Data) width of the calculation module.

[0160] like Figure 1 As shown, in some embodiments, in tasks requiring extensive parallel data processing, the data bit width of a single vector register can be equal to the data bit width of the SIMD, using vector register set 22 to store multiple data elements and cooperating with the SIMD execution unit to achieve parallel computation. The SIMD execution unit can perform the same operation on multiple data in vector register set 22 simultaneously under the control of a single instruction, accelerating data processing.

[0161] When it is necessary to select the largest or smallest k data points (top-k) from m data points, where 1 = < k < m, for scalar processor architectures, sorting algorithms such as bubble sort are usually used to compare and swap the m elements in turn, and finally select the required top-k data points from the original m data points. However, this scalar processor is a serial data processing architecture with low data processing efficiency. As m increases, the processing time will increase sharply.

[0162] like Figure 11 As shown, for vector processors using the SIMD architecture, the data bit width processed by a single instruction is very large (e.g., 128 bits). When the precision of each data is 8 bits, 16 bits, or 32 bits, a single instruction of the processor can process 16, 8, or 4 data simultaneously, significantly improving data processing efficiency.

[0163] Although a single instruction in a SIMD architecture can process multiple data points in parallel, these data points must be independent of each other. If dependencies exist, these data points cannot be executed in parallel within a single instruction. Furthermore, while SIMD architecture provides a large data processing bit width, offering the hardware foundation for parallel execution of multiple data points, the actual number of data points that can be executed in parallel may be less than the maximum number of parallel data points in a SIMD architecture due to data dependencies. This leads to reduced utilization of the SIMD execution unit; for example, in practice, only four data points (precision = 16 bits) can be processed in parallel, resulting in low hardware utilization. If this happens, the parallel computing performance of the SIMD execution unit will decrease by 50%. Therefore, to ensure the efficiency of top-k operator execution, it is crucial to study the parallel execution strategy of data on the SIMD architecture.

[0164] Based on for Figure 5 The features of the parity merge sort network described herein, wherein the calculation module performs parity merge sort on each data element in the first data vector, may include:

[0165] Step S211: Map the data elements in the first data vector to the vector register group according to the first mapping rule, and use the single instruction multiple data execution unit to perform parallel comparison-exchange operations on the data elements at the same position in the vector register group to obtain two first sub-data vectors. Then, reverse map the data elements in the two first sub-data vectors back to the address space of the storage module storing the first data vector according to the first mapping rule, so that the first data vector stored in the storage module is updated to the second data vector.

[0166] In step S211, the odd-even merge sort algorithm is used to sort the data elements in each first data vector, see [link to step S211]. Figure 5 For each first data vector, the comparison-swap operations within each parallel block of the corresponding odd-even merge sort network can be executed in parallel. To further explore the data parallelism capability of the top-k operator, the comparison-swap operations within the same parallel block can be mapped to a SIMD operation. For example... Figure 12 As shown, the two operands of the comparison-swap operation within the same parallel block of an even-odd merge sort network are mapped to two different vector registers, which makes it easier for the SIMD execution unit to perform parallel processing on the vector registers using a single instruction.

[0167] The complete execution steps of a single compare-swap operation can be found in the corresponding description above.

[0168] Step S212: Map the data elements in the second data vector to the vector register group according to the second mapping rule, and use the single instruction multiple data execution unit to perform parallel comparison-swapping operations on the data elements at the same position in the vector register group to obtain two second sub-data vectors. Then, reverse map the data elements in the two second sub-data vectors back to the address space of the storage module storing the second data vector according to the second mapping rule, so that the second data vector stored in the storage module is updated to the third data vector.

[0169] The execution process of step S212 is similar to that of step S211, the difference being that in step S212, data elements are mapped and inversely mapped according to the second mapping rule. Figure 12 Taking the odd-even merge sort network in the upper left corner as an example, such as Figure 12 As shown, for a first data vector, during the first comparison and exchange operation on the data elements in the first data vector, the data elements to be compared and exchanged in parallel block 0 are mapped to two vector registers respectively. During the second comparison and exchange operation, the data elements to be compared and exchanged in parallel block 1 are mapped to two vector registers respectively. In other words, the difference between steps S212 and S211 lies only in the different data elements to be operated on, hence the corresponding data mapping rules are also different. Their execution processes are similar, therefore the execution process of step S212 will not be described in detail here; please refer to the detailed description of the execution process of step S211 above.

[0170] Step S213: Continue in this manner until the comparison-exchange operation required for one round of parity merge sort is completed, that is, the parity merge sort of the first data vector is completed, and a first ordered data vector is obtained.

[0171] Step S213: When sorting the data elements in a data vector, the execution process of each comparison-swap operation is similar, only the mapping rule used each time is different. After completing the comparison-swap operations required for one round of parity merge sort, the parity merge sort of the first data vector is completed, resulting in a first ordered data vector.

[0172] In some embodiments, mapping the data elements in the i-th data vector to the vector register group according to the i-th mapping rule includes:

[0173] Step S231: Based on the odd-even merge sort algorithm, determine the data element pairs in the i-th data vector that need to be compared and swapped, where i=1,2.

[0174] Step S232: Map the two data elements in each data element pair to the same position in two different vector registers, wherein the data elements in each data element pair are mapped to the two different vector registers.

[0175] For example, Figure 12 The parity merge sorting network in the upper left corner illustrates the parity merge sorting process of a first data vector. First, based on the parity merge sorting algorithm, the pairs of data elements in the original first data vector that need to be compared and swapped (see parallel block 0) are determined to be (d0, d1), (d2, d3), (d4, d5), and (d6, d7). At this time, d0, d1, d2, d3, d4, d5, d6, and d7 represent the data elements with indices 0, 1, 2, 3, 4, 5, 6, and 7 in the first data vector, respectively. Next, the two data elements in each pair are mapped to two different vector registers. Figure 12 The upper right corner corresponds to the two vector registers of SIMD0. Then, based on the SIMD0 operation, a comparison-swapping operation is performed on the data elements at the same position in the two vector registers. After completing the comparison-swapping operation, two first sub-data vectors are obtained. After the data elements in these two first sub-data vectors are reverse-mapped back to the memory module, the second data vector is obtained.

[0176] Next, based on the odd-even merge sort algorithm, the data element pairs in the second data vector that need to be compared and swapped (see parallel block 1) are determined to be (d0, d2), (d1, d3), (d4, d6), and (d5, d7). At this time, d0, d1, d2, d3, d4, d5, d6, and d7 represent the data elements with indices 0, 1, 2, 3, 4, 5, 6, and 7 in the second data vector, respectively. Then, the two data elements in each data element pair are mapped to two different vector registers. Figure 12 The top right corner corresponds to the two vector registers of SIMD1. Then, based on SIMD1 operations, a comparison-swapping operation is performed on the data elements at the same position in the two vector registers. After completing the comparison-swapping operation, two second sub-data vectors are obtained. After the data elements in these two second sub-data vectors are reverse-mapped back to the memory module, a third data vector is obtained.

[0177] And so on.

[0178] In some embodiments, the step of reverse mapping the data elements in the two i-th sub-data vectors back to the address space where the storage module stores the i-th data vector according to the i-th mapping rule, so that the i-th data vector stored in the storage module is updated to the (i+1)-th data vector, includes: determining the index value of each data element according to the position of each data element in the two vector registers, writing each data element in the two i-th sub-data vectors back to the storage module according to the index value, overwriting the data element with the corresponding index value of the i-th data vector stored in the storage module, so that the i-th data vector stored in the storage module is updated to the (i+1)-th data vector.

[0179] Continue with Figure 12 For example, suppose the original first data vector is (1,3,5,2,4,8,6,7), and the goal is to obtain the k largest data elements. After the SIMD0 operation, the two first sub-data vectors obtained are (3,5,8,7) and (1,2,4,6). Next, determine the index of data element 3 in the first sub-data vector (3,5,8,7) as 0, the index of data element 5 as 1, the index of data element 8 as 2, and the index of data element 7 as 3. In the first sub-data vector (1,2,4,6), determine the index of data element 1 as 4, the index of data element 2 as 5, the index of data element 4 as 6, and the index of data element 6 as 7. Write each data element in the two first sub-data vectors back to the storage module according to the index value, overwriting the data element with the corresponding index value of the first data vector stored in the storage module, so that the first data vector stored in the storage module is updated to the second data vector (3,5,8,7,1,2,4,6).

[0180] Next, the data elements in the second data vector (3,5,8,7,1,2,4,6) are mapped to two vector registers and a SIMD1 operation is performed to obtain two second sub-data vectors (8,7,3,5) and (4,6,1,2). Then, the indices of data element 8 in the second sub-data vector (8,7,3,5) are determined to be 0, 7 to be 1, 3 to be 2, and 5 to be 3. The indices of data element 4 in the second sub-data vector (4,6,1,2) are determined to be 4, 6 to be 5, 1 to be 6, and 2 to be 7. Each data element in the two second sub-data vectors is written back to the storage module according to its index value, overwriting the data element with the corresponding index value of the second data vector stored in the storage module, so that the second data vector stored in the storage module is updated to the third data vector (8,7,3,5,4,6,1,2).

[0181] Then, the data elements in the third data vector (8,7,3,5,4,6,1,2) that need to be compared and swapped (see parallel block 2) are mapped to two vector registers, one storing vector (7,6) and the other storing (3,1). Then, a SIMD2 operation is performed to obtain two second sub-data vectors (7,6) and (3,1). Next, the index of data element 7 in the third sub-data vector (7,6) is determined to be 1, the index of data element 6 to be 5, the index of data element 3 in the third sub-data vector (3,1) to be 2, and the index of data element 1 to be 6. Each data element in the two third sub-data vectors is written back to the storage module according to its index value, overwriting the data element with the corresponding index value of the third data vector stored in the storage module, so that the third data vector stored in the storage module is updated to the fourth data vector (8,7,3,5,4,6,1,2).

[0182] And so on, eventually we get an ordered vector (8,7,6,5,4,3,2,1).

[0183] In some embodiments, when the computing module processes at least two of the first data vectors in parallel, the single instruction multiple data execution unit uses a single instruction multiple data operation instruction to execute in parallel the comparison-swap operation that can be executed in parallel during the parity merge sorting process of the at least two first data vectors.

[0184] For details, see Figure 12 If the SIMD width of the computation module (e.g., a vector processor) allows, data from the first parallel block of two different parity merge sort networks (corresponding to two first data vectors) can be mapped to a single SIMD operation, data from the second parallel block of the same two parity merge sort networks can be mapped to another SIMD operation, and so on. It is understood that, if the SIMD width of the computation module allows, data from the first parallel block of three or more parity merge sort networks can also be mapped to a single SIMD operation, data from the second parallel block can be mapped to another SIMD operation, and so on. The data mapping scheme proposed in this embodiment facilitates the efficient implementation of top-k operators based on a SIMD parallel processing architecture.

[0185] In some embodiments, the calculation module performs merge sorting on the data elements in the plurality of first ordered data vectors to obtain the top-k data elements among the m data elements, including:

[0186] Step S241: Group the plurality of first ordered data vectors into pairs, and merge and sort the data elements in the two first ordered data vectors in each group to obtain a plurality of first target data vectors;

[0187] In S241, after merging and sorting two ordered data vectors in a set, a target data vector is obtained, in which at least the first b data elements are ordered, or all data elements in the target data vector are ordered.

[0188] Step S242: Extract the top-b data elements of each of the first target data vectors to generate multiple second ordered data vectors, where b is an integer greater than or equal to the value of k, which is a power of 2.

[0189] In S242, the integer b can be the same as the grouping granularity of the original data vector, or it can be different.

[0190] Step S243: Group the plurality of second ordered data vectors into pairs, and merge and sort the data elements of the two second ordered data vectors in each group to obtain a plurality of second target data vectors;

[0191] Step S244: Extract the top-b data elements of each of the second target data vectors to generate multiple third ordered data vectors;

[0192] Step S245: Continue in this manner until an nth ordered data vector is generated;

[0193] Step S246: Extract the top-k data elements from the nth ordered data vector, wherein the top-k data elements in the nth ordered data vector are the top-k data elements among the m data elements.

[0194] Specifically, in each merge sort loop, multiple ordered data vectors are first grouped into pairs, and then the two ordered data vectors in each group are merge sorted to obtain the target data vector. Then, the top-b data elements of each target data vector are extracted to obtain a new ordered data vector. This loop is executed until the number of ordered data vectors obtained is equal to 1. The first k data are extracted from the ordered data vector, which are the final selected largest or smallest k data.

[0195] In some embodiments, the calculation module groups the plurality of i-th ordered data vectors into pairs, and merges and sorts the data elements in the two i-th ordered data vectors within each group to obtain a plurality of i-th target data vectors, including:

[0196] The plurality of i-th ordered data vectors are grouped pairwise, and each time a group of i-th ordered data vectors is processed, the merge sort of the two i-th ordered data vectors in each group is completed sequentially, and finally a plurality of i-th target data vectors are obtained; or,

[0197] The plurality of i-th ordered data vectors are grouped in pairs, and at least two groups of i-th ordered data vectors are processed in parallel each time until the merging and sorting of the two i-th ordered data vectors in each group is completed, and finally a plurality of i-th target data vectors are obtained, where i=1,2,3,…,n-1.

[0198] Specifically, in each merge sort loop, multiple ordered data vectors need to be grouped into pairs. Then, the data elements in the two ordered data vectors within each group are merge sorted. When merging sorting the data elements in the two ordered data vectors within each group, one group of ordered data vectors can be processed at a time to complete the merge sorting of the two ordered data vectors within each group sequentially. Alternatively, at least two groups of ordered data vectors can be processed in parallel each time until the merge sorting of the two ordered data vectors within each group is completed.

[0199] In some embodiments, the calculation module performs merge sorting on the data elements of the two i-th ordered data vectors within each group, including:

[0200] S251. Map the data elements in the two i-th ordered data vectors in the group to the vector register group according to the third mapping rule. Use the single instruction multiple data execution unit to perform parallel comparison-exchange operations on the data elements at the same position in the vector register group to obtain two first intermediate data vectors. Then, reverse map the data elements in the two first intermediate data vectors back to the storage module according to the third mapping rule to obtain two second intermediate data vectors.

[0201] Step S251 involves using a conventional merge sort algorithm to map the data elements of the two i-th ordered data vectors within a group to two vector registers, with one i-th ordered data vector mapped to one vector register and the other to the other. Then, the single instruction multiple data execution unit (SIM) performs parallel comparison-swap operations on the data elements at the same positions in the two vector registers to obtain two first intermediate data vectors. Next, the data elements of these two first intermediate data vectors are written back to the storage module to obtain two second intermediate data vectors. The data elements of the two first intermediate data vectors can be written back to the address space in the storage module where the two i-th ordered data vectors are stored. At this point, the storage module updates the two i-th ordered data vectors to the two second intermediate data vectors, which can be identical to the two first intermediate data vectors.

[0202] Taking the merge sort of two sets of 8-element data as an example, when k=7, the conventional merge sort algorithm merges and sorts the ordered sequences of the two sets of 8-element data. The goal is to output a complete ordered sequence of 16 elements. By performing a complete comparison-exchange operation on all elements, the overall orderliness of the sequence is ensured (regardless of whether these comparison-exchange operations are related to the top-7 output sequence), so that all 16 elements in the final output are ordered.

[0203] However, in the top-k problem, it is usually only necessary to ensure that the first k data in the output result are ordered sequences, while the conventional merge sort aims to make the entire output result ordered. Therefore, the efficiency of the top-k operator can be improved by pruning some comparison-swap operations that do not contribute to the top-k output, reducing redundant processing logic. Figure 13 This is a comparison diagram of regular merge sort and optimized merge sort. In the scenario of k=7, since it is only necessary to ensure that the first k elements in the output are ordered, the regular merge sort is eliminated by identifying and retaining the comparison-exchange operations related to the first 7 elements. Figure 13 The comparison-swap operations on the left that do not contribute to the top-7, such as... Figure 13 Nine comparison-swap operations are pruned, namely (7,15), (7,11), (7,9), (7,8), (10,12), (11,13), (9,10), (11,12), and (13,14), to avoid invalid processing of invalid elements and thus improve the efficiency of operator execution.

[0204] Therefore, in step S251, the data elements in the two i-th ordered data vectors within a group can be mapped to the two vector registers according to the mapping rules of the optimized merge sort algorithm. The subsequent comparison-swap operation, reverse mapping, and write-back operation are similar to the conventional merge sort operation.

[0205] S252. Map the data elements in the two second intermediate data vectors to the vector register group according to the fourth mapping rule. Use the single instruction multiple data execution unit to perform parallel comparison-exchange operations on the data elements at the same position in the vector register group to obtain two third intermediate data vectors. Then, reverse map the data elements in the two third intermediate data vectors back to the storage module according to the fourth mapping rule to obtain two fourth intermediate vectors.

[0206] The execution process of step S252 is similar to that of step S251, and will not be described again here.

[0207] S253. Continue in this manner until the comparison-exchange operation required for one round of merge sort is completed, that is, the merge sort of the two i-th ordered data vectors is completed to obtain an i-th target data vector, where i=1,2,3,…,n-1.

[0208] Specifically, after completing the comparison-swap operations required for one round of merge sort, two data vectors are obtained. These two data vectors are then concatenated to obtain the i-th target data vector. For example, if the desired top-k data elements are the k largest data elements among m data elements, then during concatenation, the data vector with the larger data element is placed first, and the data vector with the smaller data element is placed last. If the desired top-k data elements are the k smallest data elements among m data elements, then during concatenation, the data vector with the smaller data element is placed first, and the data vector with the larger data element is placed last.

[0209] It should be understood that if the conventional merge sort algorithm is used to merge and sort the data elements of the two i-th ordered data vectors in each group, then the data elements in the i-th target data vector obtained in step S253 above are an ordered sequence; if the optimized merge sort algorithm is used to merge and sort the data elements of the two i-th ordered data vectors in each group, then at least the first b data elements in the i-th target data vector obtained in step S253 above are ordered.

[0210] In some embodiments, mapping the data elements of the two i-th ordered data vectors within the group to the vector register group according to the third mapping rule includes:

[0211] Step S271: Based on the set merge sort algorithm, determine the first valid data element pair that needs to be compared and swapped between the two i-th ordered data vectors in the group.

[0212] Step S272: Map the two data elements in each of the first valid data element pairs to the same location in two different vector registers, wherein the data elements in each of the first valid data element pairs are mapped to the two different vector registers.

[0213] Specifically, the set merge sort algorithm can be a conventional (or classic) merge sort algorithm, or the optimized merge sort algorithm described above. For each round of merge sorting, after grouping the ordered data vectors, the data elements in the two ordered data vectors within each group are mapped to two vector registers according to the set merge sort algorithm. The third mapping rule is described in detail below through an example. Assume that the two i-th ordered data vectors in a group are (18,15,13,11,9,7,8,3) and (20,16,13,9,8,7,6,0), the goal is to obtain the top-7 data, and the set merge sort algorithm is the optimized merge sort algorithm. Then, see... Figure 13 The first valid data element pairs that need to be compared and swapped in the two i-th ordered data vectors are (18,20), (15,16), (13,13), (11,9), (9,8), (7,7), and (8,6). The data elements 3 and 0, which are in the 8th position in the two ordered data vectors, do not contribute substantially to the filtering of the top-7 data, so no comparison and swap operation is required for them. After determining the first valid data element pairs that need to be compared and swapped, the two data elements in each first valid data element pair are mapped to the same position in two different vector registers. Then, a SIMD operation is used to perform a parallel comparison and swap operation on the two data elements in each first valid data element pair. The two first intermediate data vectors are (20,16,13,11,9,7,8) and (18,15,13,9,8,7,6).

[0214] In some embodiments, the step of reverse mapping the data elements in the two first intermediate data vectors back to the storage module according to the third mapping rule to obtain two second intermediate data vectors includes: determining the index value of each data element according to the position of each data element in the two vector registers, writing each data element in the two first intermediate data vectors back to the storage module according to the index value, overwriting the data elements corresponding to the index values ​​of the two i-th ordered data vectors stored in the storage module, so that the two i-th ordered data vectors stored in the storage module are updated to two second intermediate data vectors.

[0215] Specifically, taking the two first intermediate data vectors (20,16,13,11,9,7,8) and (18,15,13,9,8,7,6) as examples, based on the storage location of each data element, the indices of each data element in the first intermediate data vector (20,16,13,11,9,7,8) from left to right are 0, 1, 2, 3, 4, 5, 6, and the indices of each data element in the first intermediate data vector (18,15,13,9,8,7,6) from left to right are 8, 9, 10, 11, 12, 13, 14. Then, the data elements in the first intermediate data vector (20,16,13,11,9,7,8) are arranged according to their indices. Write back to the storage module, overwriting the first 7 data elements of the i-th ordered data vector (18,15,13,11,9,7,8,3) in the storage module. At this time, the i-th ordered data vector (18,15,13,11,9,7,8,3) in the storage module is updated to the second intermediate data vector (20,16,13,11,9,7,8,3). Similarly, after the first intermediate data vector (18,15,13,9,8,7,6) is written back to the storage module, the i-th ordered data vector (20,16,13,9,8,7,6,0) in the storage module is updated to the second intermediate data vector (18,15,13,9,8,7,6,0).

[0216] Then, the data elements in the two second intermediate data vectors are mapped to the vector register group according to the fourth mapping rule. In some embodiments, mapping the data elements in the two second intermediate data vectors to the vector register group according to the fourth mapping rule includes:

[0217] Step S291: Based on the set merge sort algorithm, determine the second effective data element pair that needs to be compared and swapped in the two second intermediate data vectors;

[0218] Step S292: Map the two data elements in each of the second valid data element pairs to the same position in two different vector registers, wherein the data elements in each of the second valid data element pairs are mapped to the two different vector registers.

[0219] Specifically, taking the two second intermediate data vectors (20,16,13,11,9,7,8,3) and (18,15,13,9,8,7,6,0) as examples, see [link to relevant documentation]. Figure 13Based on the optimized merge sort algorithm, the second valid data element pairs that need to be compared and swapped in the two second intermediate data vectors are determined to be (9, 18), (7, 15), and (8, 13). Then, the two data elements in each data element pair are mapped to the same position in two different vector registers. Next, a SIMD operation is used to compare and swap the data elements at the same position in the vector registers to obtain two third intermediate data vectors (18, 15, 13) and (9, 7, 8).

[0220] Next, the two third intermediate data vectors are reverse-mapped back to the storage module according to the fourth mapping rule to obtain two fourth intermediate vectors. In some embodiments, the step of reverse-mapping the two third intermediate data vectors back to the storage module according to the fourth mapping rule to obtain two fourth intermediate vectors includes: determining the index value of each data element in the two third intermediate data vectors according to the position of each data element in the two vector registers, writing each data element in the two third intermediate data vectors back to the storage module according to the index value, overwriting the data elements of the two second intermediate data vectors stored in the storage module corresponding to the index values, so that the two second intermediate data vectors stored in the storage module are updated to two fourth intermediate data vectors.

[0221] Specifically, continuing with the two third intermediate data vectors (18,15,13) and (9,7,8) mentioned above, the indices of each data element in the third intermediate data vector (18,15,13) from left to right are (4,5,6), and the indices of each data element in the third intermediate data vector (9,7,8) from left to right are (8,9,10). The third intermediate data vector (18,15,13) is written back to the storage module according to the indices of each data element, overwriting the data elements of the second intermediate data vector (20,16,13,11,9,7,8,3) stored in the storage module with the corresponding index values, so that the second intermediate data vector (20,16,13,11,9,7,8,3) stored in the storage module is updated to the fourth data vector (20,16,13,11,9,7,8,3). 18,15,13,3); Similarly, the third intermediate data vector (9,7,8) is written back to the storage module, and the second intermediate data vector (18,15,13,9,8,7,6,0) stored in the storage module is updated to the fourth data vector (9,7,8,9,8,7,6,0).

[0222] This process continues until the comparison-swap operations required for one round of merge sort are completed, resulting in two data vectors: (20,18,16,15,13,13,11,3) and (9,7,8,9,8,7,6,0). These two data vectors are then concatenated to obtain a target data vector of the i-th order: (20,18,16,15,13,13,11,3,9,7,8,9,8,7,6,0). It can be seen that the first 8 elements in this target data vector of the i-th order are ordered, and the 7 largest elements are at the beginning.

[0223] In some embodiments, when the computing module processes at least two sets of the i-th ordered data vectors in parallel each time, the single instruction multiple data execution unit uses a single instruction multiple data operation instruction to execute the comparison-swap operation that can be executed in parallel during the merge sorting process of the two sets of i-th ordered data vectors.

[0224] For example, the process of merging and sorting four pairs of ordered numerical sequences and position index sequences of 8 elements is as follows:

[0225] In the first round of merge sort, a large number of comparison-swap operations need to be performed on the four ordered sequences. These comparison-swap operations within the same parallel block can be executed in parallel. Therefore, according to the granularity of the parallel block, these comparison-swap operations are integrated into a set of SIMD operations, such as... Figure 26 As shown, the comparison-swapping operations of the four ordered sequences can be integrated into five SIMD operations. For each SIMD operation, the source operands (the data vector to be compared and the position index vector) are read from the storage module, and the data (data vector and position index vector) after the comparison-swapping operation is completed are written back to the storage module. The five SIMD operations are executed sequentially, resulting in two sets of 16-element sequences. The first eight elements of each sequence are extracted, namely d0~d7 and d16~d23, resulting in two sets of 8-element ordered numerical sequences and position index sequences.

[0226] The second round of merge sort performs merge sort on the two previously obtained ordered sequences of 8 elements, such as... Figure 27 As shown, these comparison-swap operations are integrated according to the parallel block granularity to obtain 4 SIMD operations. The 4 SIMD operations are executed sequentially, and after completion, a set of 16-element data sequence and index sequence are obtained. The first 8 values ​​of the sequence are extracted, and the top-8 numerical results and position indices are output.

[0227] The RISC-V instruction set architecture, as an open-source, scalable, and completely free ISA (Instruction Set Architecture), provides a new path that balances flexibility and performance for the efficient implementation of top-k operators. Its core advantage lies in achieving hardware-level acceleration of the data top-k filtering process by avoiding reliance on dedicated hardware through modular instruction extensions and standardized toolchains. The data processing device provided in this application embodiment can implement a top-k operator implementation scheme based on the RISC-V vector extension architecture. This scheme can integrate various standard RISC-V vector extension instructions based on the RISC-V instruction set architecture to achieve parallel execution of top-k operators. With the popularization of standardized acceleration instructions such as vector extension (RVV) and the continuous maturation of open-source toolchains and ecosystems, this scheme is expected to become a core technology for future intelligent computing (from real-time filtering at the edge to large-scale sparse activation in the cloud).

[0228] Figure 14 This is a schematic diagram of the structure of a data processing device provided in another embodiment of this application. Figure 14 It demonstrates the complete process of a data processing device, from instruction fetching, decoding, and scheduling to achieving efficient parallel computing using SIMD execution units.

[0229] like Figure 14 As shown, in some embodiments, the data processing device 100 further includes:

[0230] The instruction decoding unit 3 is coupled to the storage module 1 and is configured to parse the RISC-V instructions extracted from the storage module 1. The instruction decoding unit 3 can quickly parse these instructions, rapidly identify the type of operation to be performed by the instruction and the source of operands, and prepare for subsequent computation and processing.

[0231] Scheduler 4, coupled to the instruction decoding unit 3 and the computing module 2, is configured to determine the execution order of each RISC-V instruction based on the instruction characteristics of the RISC-V instructions and the resource status of the computing module 2, so that the single instruction multiple data execution unit 21 executes the RISC-V instructions in the order they are executed. Scheduler 4 rationally arranges the instruction execution order according to the resource status (such as register resources, execution unit status, etc.) and instruction characteristics under different application scenarios.

[0232] In some embodiments, the storage module 1 includes a static random access memory (SRAM) coupled to the computing module 2 and configured to store data and RISC-V instructions. Static random access memory (SRAM) has extremely high data access speed and, in RISC-V architecture systems, is used to store program instructions and data to meet the architecture system's requirements for data access speed.

[0233] like Figure 14 As shown, in some embodiments, the data processing apparatus 100 further includes:

[0234] Instruction cache unit (instruction cache) 5 is coupled to the static random access memory and is configured to retrieve and store RISC-V instructions from the static random access memory. The instruction cache is a high-speed cache space. The RISC-V architecture emphasizes the efficiency of memory access. It usually retrieves frequently used instructions from SRAM quickly and stores them in the instruction cache to reduce the time the processor waits for instructions.

[0235] The instruction fetch cache unit 6, coupled to the instruction cache unit 5 and the instruction decoding unit 3, is configured to store RISC-V instructions fetched from the instruction cache unit 5 in an orderly manner, enabling the instruction decoding unit 3 to fetch RISC-V instructions from the instruction fetch cache unit 5. The instruction fetch cache unit 6 further temporarily stores instructions fetched from the instruction cache. During the instruction fetch phase, the RISC-V architecture processor uses the instruction fetch cache unit 6 to store instructions in an orderly manner, preparing for subsequent decoding and execution.

[0236] In addition, the RISC-V architecture uses vector register sets to store data vectors. In tasks requiring a large amount of parallel data processing, the data bit width of a single vector register can be equal to the data bit width of SIMD, using vector register sets to store multiple data elements and working in conjunction with the SIMD execution unit 21 to achieve parallel computation.

[0237] The RISC-V Vector Extensions instruction set supports SIMD operations. The SIMD execution unit 21 can perform the same operation on multiple data elements in a vector register set simultaneously under the control of a single instruction, accelerating data processing. RVV supports Single Instruction Multiple Data (SIMD) mode, allowing comparison-swap operations to be performed on multiple data elements in parallel within a single instruction cycle. For example, traditional scalar instructions process vector elements one by one, while RVV's compare and merge instructions can process multiple element pairs simultaneously, significantly reducing the number of instruction executions and greatly improving data processing speed, making it particularly suitable for large-scale data sorting and filtering scenarios.

[0238] To better understand this application, the following detailed description of the method executed by the data processing apparatus provided in this application is provided through several specific embodiments.

[0239] Example 1

[0240] This embodiment corresponds to Figure 2The steps shown indicate that x or x1 is an integer multiple of the SIMD processing capability. Assuming a SIMD processing bit width of 128 bits and a data precision of 16 bits, a single SIMD execution unit can process 8 data items in parallel. For example, consider selecting the top-8 elements from 8 batches (groups) of data, each batch containing 32 elements:

[0241] Step 1: Group the 8 batches of raw data according to the batch, forming 8 groups of 32-element sequences. Each group consists of a numerical sequence and a position index sequence.

[0242] Step 2: Considering the need to select the top-8 elements, the 8 sets of 32-element sequences are divided into 8x8 first sub-vector matrices, resulting in 4 8x8 first sub-vector matrices. Then, odd-even merge sort is used to sort these four 8x8 first sub-vector matrices within a block:

[0243] Since a total of four 8x8 first subvector matrices need to be sorted within a block, and the sorting processes for different first subvector matrices are independent, the sorting processes for different groups (columns within the first subvector matrices) are also independent. Considering that a single SIMD operation can process 8 data points simultaneously, we choose to select two columns from each first subvector matrix to form one SIMD operation. For example... Figure 15 The diagram shows how data with the same index in each of the four first sub-vector matrices (corresponding to groups 0-7, 8-15, 16-23, and 24-31 respectively) are grouped into a vector. For example, the vector in group 0 consists of the 0th data item in each of the eight rows. Groups 0-7 are equivalent to... Figure 5 0~7 in the range, then according to Figure 5 The comparison-swap operation can be performed as shown. Figure 16 As shown, the comparison-exchange operation of four 8x8 first subvector matrices can be mapped to 76 SIMD operations. After the 76 SIMD operations are completed, four ordered 8x8 first subvector matrices (intermediate matrices) are obtained. Here, "ordered" means that the rows in the first subvector matrix are arranged in an ordered manner. Each first subvector matrix contains a sequence of 8 numerical elements and a sequence of position indices.

[0244] The execution flow of each SIMD operation described above can be as follows: Figure 17As shown: First, the source operands (the value vector and the position index vector to be compared) of the compare-swap operation are read from SRAM. Then, the RISC-V instructions control the SIMD unit to complete the compare-swap operation and save the processing result to a free register in the vector register group. Since the number of vector register groups is usually small, the output result (value vector and position index vector) after the compare-swap operation is usually written back to SRAM to release the occupied vector registers. Frequent SRAM reads and writes increase additional power consumption and latency. In fact, the output result of some SIMD operations may be the source operand of subsequent SIMD operations. Therefore, these output results can be kept in the vector registers without being written back to SRAM, waiting for subsequent SIMD operations to process. At this time, the operation combination can be classified using the method of the embodiment corresponding to steps S141 to S148. After all operations in the operation combination are completed, the output result is written back to SRAM to reduce additional power consumption and latency.

[0245] Step 3: Combine the ordered numerical sequences and position index sequences of the four 8x8 intermediate matrices pairwise to obtain two 8x16 element matrices, and then perform merge sort (executed in a loop):

[0246] 1) In the first round of merge sort, when performing intra-block merge sort on two 8x16 element matrices, the sorting processes of different blocks are independent, and therefore the sorting processes between different groups are also independent. Considering that a single SIMD operation can process 8 data points simultaneously, two columns of data can be selected from each element matrix to form a single SIMD operation; for example... Figure 18 The diagram shows how data with the same index within each element matrix (two 8-row, 16-column element matrices corresponding to groups 0-15 and 16-31 respectively) are grouped into a vector. For example, the vector for group 0 consists of the 0th data item in each of the 8 rows. Groups 0-15 (and 16-31) are equivalent to... Figure 8 0~15 in the range, then according to Figure 8 The comparison-swap operation can be performed as shown. Figure 18 As shown, the comparison-swap operation of two 8-row, 16-column element matrices can be mapped to 40 SIMD operations. After the 40 SIMD operations are completed, two 8-row, 16-column matrixes are obtained with their rows ordered. The first 8 elements of each matrix are extracted, namely d0~d7 and d16~d23, resulting in two 8-row, 8-column element matrices. Each element matrix contains an ordered sequence of 8 elements and a sequence of position indices.

[0247] 2) The second round of merge sorting merges the two previously obtained 8x8 matrix elements, first combining them pairwise to obtain a single 8x16 matrix. For example... Figure 19The diagram shows how to group the column data of an 8x16 element matrix into a vector. The comparison-swap operation of the 8x16 element matrix can be mapped to 20 SIMD operations. After the 20 SIMD operations are completed, an 8x16 element matrix with in-row ordered arrangement is obtained.

[0248] Step 4: Extract the first 8 values ​​from the 8-row, 16-column ordered matrix of elements, and output the top-8 numerical results and position index results of the 8 batches of data.

[0249] Example 2

[0250] In this embodiment, steps S401-S404 are shown, meaning that the number of subgroups after x21 is divided into groups with granularity b is an integer multiple of the SIMD processing capability. Assuming a SIMD processing bit width of 128 bits and a data precision of 16 bits, the SIMD execution unit can process 8 data items in parallel with a single instruction. For example, selecting the top-8 elements from 4 batches of data, each with 32 elements:

[0251] Step 1: Divide the 32 elements in each group into 4 groups with a granularity of 8, for a total of 16 groups of 8-element sequences. Each group consists of a numerical sequence and a position index sequence, which can form a 16-row, 8-column subvector matrix.

[0252] Step 2: Divide the above 16-row 8-column subvector matrix into two 8-row 8-column first subvector matrices, and use odd-even merge sort to sort each first subvector matrix within a block;

[0253] Considering that a single SIMD operation can process 8 data points simultaneously, two columns of data can be selected from each first subvector matrix for comparison and swapping, forming a single SIMD operation. For example... Figure 20 The diagram shows how data with the same index within the first sub-vector matrix (8 rows and 8 columns) are grouped into vectors. For example, vector group 0 is composed of the 0th data element from the 8 rows of the first sub-vector matrix. Groups 0 through 7 are equivalent to... Figure 5 0~7 in the range, then according to Figure 5 The comparison-swap operation can be performed as shown. Figure 21 As shown, the comparison-exchange operation of two 8x8 first subvector matrices can be mapped to 19*2 SIMD operations. After the 19*2 SIMD operations are completed, two ordered 8x8 first subvector matrices (intermediate matrices) are obtained. Here, "ordered" means that the rows in the first subvector matrix are arranged in an ordered manner. Each first subvector matrix contains a sequence of 8 numerical elements and a sequence of position indices.

[0254] The execution flow of each SIMD operation described above can be as follows: Figure 17As shown: First, the source operands (the value vector and the position index vector to be compared) of the compare-swap operation are read from SRAM. Then, the RISC-V instructions control the SIMD unit to complete the compare-swap operation and save the processing result to a free register in the vector register group. Since the number of vector register groups is usually small, the output result (value vector and position index vector) after the compare-swap operation is usually written back to SRAM to release the occupied vector registers. Frequent SRAM reads and writes increase additional power consumption and latency. In fact, the output result of some SIMD operations may be the source operand of subsequent SIMD operations. Therefore, these output results can be kept in the vector registers without being written back to SRAM, waiting for subsequent SIMD operations to process. At this time, the operation combination can be classified using the method of the embodiment corresponding to steps S141 to S148. After all operations in the operation combination are completed, the output result is written back to SRAM to reduce additional power consumption and latency.

[0255] Step 3: Combine the ordered numerical sequences and position index sequences of the two 8x8 intermediate matrices pairwise to obtain an 8x16 element matrix, and then perform merge sort (executed in a loop):

[0256] 1) In the first round of merge sort, when merging an 8x16 matrix, the merge sorting processes for different groups are independent. Considering that a single SIMD operation can process 8 data items simultaneously, two columns of data can be selected from the matrix for comparison and swapping to form one SIMD operation; for example... Figure 22 The diagram shows how data with the same index in an element matrix are grouped into vectors. For example, vector group 0 consists of the 0th data element from each of the 8 rows. Groups 0 through 15 are equivalent to... Figure 8 0~15 in the range, then according to Figure 9 The comparison-swap operation can be performed as shown. Figure 22 As shown, a comparison-swap operation on an 8x16 matrix can be mapped to 20 SIMD operations. The 20 SIMD operations are executed sequentially, resulting in an 8x16 matrix with in-row ordered elements. The first 8 elements are then extracted, such as d0~d7 and d16~d23, resulting in an 8x8 matrix.

[0257] 2) The second round of merge sorting involves merging the previously obtained 8x8 matrix into pairs, resulting in a 4x16 matrix. For example... Figure 23The data with the same index in the element matrix are grouped into a vector. The comparison-swap operation of a 4x16 element matrix can be mapped to 20 SIMD operations. The 20 SIMD operations are executed sequentially. After the execution, a 4x16 element matrix with in-row ordered arrangement is obtained, with each row representing a batch of data.

[0258] Step 4: Extract the first 8 values ​​from the obtained 4x16 element matrix, and output the top-8 numerical results and position index results of the 4 batches of data.

[0259] Example 3

[0260] In this embodiment, the number of subgroups after x22 is divided into segments with b as the granularity is not an integer multiple of the SIMD processing capacity, which is a small-batch data application scenario.

[0261] like Figure 24 As shown, assuming the SIMD execution unit has a processing bit width of 128 bits and a data precision of 16 bits, a single SIMD execution unit can process 8 data items in parallel. Taking the selection of the top-8 elements from a batch of 32 elements as an example:

[0262] Step 1: Divide the 32 elements into 4 groups with a granularity of 8;

[0263] Step 2: Use odd-even merge sort to sort the 4 groups of 8-element sequences within each group;

[0264] Since a single SIMD operation can process 8 data points simultaneously, and the data batch size is 1, to ensure the utilization rate of SIMD, it is necessary to analyze the parallelizability of each comparison-swap operation based on the granularity of the comparison-swap operation. For example... Figure 24 As shown, the odd-even merge sort network for an 8-element sequence contains 6 parallel blocks. The comparison-swap operations within the parallel blocks are independent and can be integrated into a single SIMD operation. However, there are dependencies between the 6 parallel blocks, which need to be executed from left to right.

[0265] Merge sorting four ordered numerical sequences requires numerous comparison-swap operations. These operations, which are within the same parallel block, can be executed in parallel. Therefore, based on the parallel block granularity, these comparison-swap operations are integrated into a set of SIMD operations, such as... Figure 25 As shown, the comparison-swapping operation of 4 ordered sequences can be mapped to 10 SIMD operations. The 10 SIMD operations are executed sequentially, and after execution, 4 sequences of 8 elements are obtained. Each sequence contains a numerical sequence and a position index sequence of 8 elements.

[0266] See Figure 17Each SIMD operation first reads the source operands (the value vector and the position index vector to be compared) of the compare-swap operation from SRAM. Then, the RISC-V instruction controls the SIMD unit to complete the compare-swap operation and save the processing result to the free register in the vector register group. Since the number of vector register groups is usually relatively small, it is usually necessary to write the output result (value vector and position index vector) after the compare-swap operation is completed back to SRAM to release the occupancy of the vector register.

[0267] Step 3: Group the four sets of 8-element ordered numerical sequences and position index sequences pairwise, and perform merge sort (execute in a loop):

[0268] In the first round of merge sort, a large number of comparison-swap operations need to be performed on the four ordered sequences. These comparison-swap operations within the same parallel block can be executed in parallel. Therefore, according to the granularity of the parallel block, these comparison-swap operations are integrated into a set of SIMD operations, such as... Figure 26 As shown, the comparison-swapping operations of the four ordered sequences can be integrated into five SIMD operations. In each SIMD operation, the source operands (the value vector and position index vector to be compared) are read from SRAM, and the data (value vector and position index vector) after the comparison-swapping operation is completed are written back to SRAM. The five SIMD operations are executed sequentially, resulting in two sequences of 16 elements each. The first eight elements of each sequence are extracted, namely d0~d7 and d... 16 ~d 23 Then, we obtain two sets of ordered numerical sequences and position index sequences of 8 elements each.

[0269] The second round of merge sort performs merge sort on the two previously obtained ordered sequences of 8 elements, such as... Figure 27 As shown, these comparison-swap operations are integrated according to the parallel block granularity to obtain 4 SIMD operations. The 4 SIMD operations are executed sequentially, and after completion, a set of 16-element numerical sequence and index sequence is obtained. The first 8 values ​​of the sequence are extracted, and the top-8 numerical results and position indices are output.

[0270] Step 4: Extract the first 8 values ​​from the obtained 16-element numerical sequence and index sequence, and output the top-8 numerical results and position index results of a batch of data.

[0271] Based on the same inventive concept, embodiments of this application also provide a data processing method. Figure 28 This is a flowchart illustrating a data processing method provided in an embodiment of this application, as shown below. Figure 28 As shown in the embodiment of this application, a data processing method includes:

[0272] Step S281: Access the x sets of original data vectors stored in the storage module, wherein the original data vectors contain m data elements, and m is a positive integer;

[0273] Step S282: Determine whether x is an integer multiple of the SIMD processing capability. If it is an integer multiple, proceed to step S283; otherwise, proceed to step S284.

[0274] Step S283: Based on the first data processing strategy, load the x groups of original data vectors into the vector register group, and use the single instruction multiple data execution unit to perform odd-even merge sort and merge sort to obtain the top-k data elements in the x groups of original data vectors, where k is a positive integer.

[0275] Step S284: Split x into x1 and x2, where x1 is an integer multiple of the SIMD processing capability and x2 is the remainder. Based on the first data processing strategy, load the x1 group of original data vectors into the vector register group. Use the single instruction multiple data execution unit to perform odd-even merge sort and merge sort to obtain the top-k data elements in the x1 group of original data vectors.

[0276] Step S285: Determine whether x2 can be split into x21 and x22, where x21×n is an integer multiple of the SIMD processing capability, n is the number of subgroups obtained after splitting m data elements according to b granularity, and x22 is the remaining part; if x21 is not 0, proceed to step S286, if x22 is not 0, proceed to step S287.

[0277] Step S286: Based on the second data processing strategy, load x21 sets of original data vectors into the vector register group, and use the single instruction multiple data execution unit to perform odd-even merge sort and merge sort to obtain the top-k data elements in x21 sets of original data vectors;

[0278] Step S287: Based on the third data processing strategy, load the x22 original data vectors into the vector register group, and use the single instruction multiple data execution unit to perform odd-even merge sort and merge sort to obtain the top-k data elements in the x22 original data vectors.

[0279] The embodiments of the method provided in this application are specifically the data processing flow of the data processing device in the above-described device embodiments. The specific implementation method will not be repeated here, but can be referred to the detailed description of the above-described device embodiments.

[0280] The data processing method proposed in this application can select different processing methods for odd-even merge sort and merge sort according to the number of input original data vectors and the processing capability of SIMD, thereby obtaining the corresponding number of top-k data elements. The algorithm of this application is flexible, does not require a dedicated hardware sorting network, avoids dependence on dedicated hardware, and improves programming flexibility and versatility, which is conducive to the expansion and migration of large models on different hardware platforms.

[0281] This application also provides a chip that includes the data processing device described in any of the above embodiments. For example, the chip may be a many-core artificial intelligence (AI) chip based on the RISC-V vector extended instruction architecture.

[0282] The chip provided in this application embodiment includes the data processing device, and therefore can achieve the same technical effect as the data processing device, which will not be described in detail here.

[0283] This application also provides an electronic device, which includes the chip described in any of the above embodiments.

[0284] The electronic device provided in this application embodiment includes the data processing device, and therefore can achieve the same technical effect as the data processing device, which will not be described in detail here.

[0285] Figure 29 This is a block diagram of an electronic device provided in an embodiment of this application. For example, the electronic device 200 may be an AI server, an artificial intelligence server cluster, a training and promotion integrated machine, etc.

[0286] Reference Figure 29 The electronic device 200 may include one or more of the following components: an AI-accelerated computing module, a CPU module, a power supply module, a hard drive module, and a fan module. Each module works in conjunction with the bus system through a standardized hardware interface, with the specific architecture as follows:

[0287] The AI-accelerated computing module comprises multiple AI accelerator cards deployed in parallel. Each AI accelerator card integrates at least one AI accelerator chip (such as an RPU chip, GPU chip, or CGRA chip). Data communication between the AI ​​accelerator cards is achieved through a high-speed card-to-card (C2C) interconnect structure, supporting low-latency, high-bandwidth horizontal scaling. The AI ​​accelerator chip is dedicated to performing AI computing tasks such as high-density matrix operations, neural network model training, and / or inference, providing the main computing power support.

[0288] The CPU module includes at least one CPU board, which houses a central processing unit (CPU) chip and associated CPU memory (such as DDR4 / DDR5, RAM). The CPU chip serves as the system control center, responsible for task scheduling, resource allocation, I / O management, and coordinating the parallel computing of the AI ​​acceleration computing module, while also handling non-accelerated general-purpose computing tasks.

[0289] The power module is equipped with redundant power supply units to provide stable power distribution and management for the AI ​​acceleration computing module, CPU module and other modules.

[0290] The hard drive module integrates a high-speed solid-state drive (SSD) and / or a large-capacity hard disk drive (HDD), connected to the system bus via a backplane. The hard drive stores the operating system, AI training datasets, model parameters, and computation results, providing high-throughput data read / write channels and supporting data preprocessing and persistence.

[0291] The fan module uses a multi-zone independent speed-controlled fan array, which is configured in key heat source areas (such as AI accelerator cards and CPU heat dissipation areas) to achieve system heat dissipation through forced air cooling and ensure the stable operation of high-efficiency computing components.

[0292] The CPU module is connected to the AI ​​acceleration computing module via the PCIe bus to enable task distribution, result collection, and memory coordination.

[0293] The CPU module manages the data access of the hard drive module through SATA / SAS / NVMe interfaces.

[0294] The power module provides tiered power to all functional modules through the power distribution backplane.

[0295] The fan module adjusts the fan speed based on temperature monitoring signals from the CPU board and AI accelerator card.

[0296] This embodiment of the application provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions, and when the program instructions are executed by a computer, the computer can perform the methods provided in the above-described method embodiments.

[0297] This embodiment provides a computer-readable storage medium storing a computer program that causes the computer to perform the methods provided in the above-described method embodiments.

[0298] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0299] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0300] In the description of this specification, the references to terms such as "an embodiment," "a specific embodiment," "some embodiments," "for example," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0301] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of this application. It should be understood that the above descriptions are merely specific embodiments of this application and are not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A data processing apparatus, characterized in that, include: The storage module is configured to store x sets of original data vectors, each original data vector containing m data elements, where x and m are positive integers; The computing module includes a single instruction multiple data (SMI) unit and a vector register set, wherein the SMI is coupled to the vector register set, and the vector register set is coupled to the memory module. The computing module is configured as follows: Determine if x is an integer multiple of the SIMD processing capability; In response to x being an integer multiple of the SIMD processing capability, the x sets of original data vectors are loaded into the vector register group based on the first data processing strategy, and the single instruction multiple data execution unit is used to perform odd-even merge sort and merge sort to obtain the top-k data elements in the x sets of original data vectors, where k is a positive integer; In response to x not being an integer multiple of the SIMD processing capability, x is split into x1 and x2, where x1 is an integer multiple of the SIMD processing capability and x2 is the remainder. Based on the first data processing strategy, the x1 group of original data vectors is loaded into the vector register group, and the single instruction multiple data execution unit is used to perform odd-even merge sort and merge sort to obtain the top-k data elements in the x1 group of original data vectors. Determine whether x2 can be split into x21 and x22, where x21×n is an integer multiple of the SIMD processing capability, n is the number of subgroups obtained after splitting m data elements according to b granularity, b is an integer greater than or equal to the power of 2 of k, and x22 is the remaining part; In response to x21 being non-zero, the original data vector of group x21 is loaded into the vector register group based on the second data processing strategy, and the single instruction multiple data execution unit is used to perform odd-even merge sort and merge sort to obtain the top-k data elements in the original data vector of group x21. In response to x22 being non-zero, the x22 original data vectors are loaded into the vector register group based on the third data processing strategy. The single instruction multiple data execution unit is then used to perform odd-even merge sort and merge sort to obtain the top-k data elements in the x22 original data vectors.

2. The data processing apparatus according to claim 1, characterized in that, The calculation module loads x groups or x1 groups of original data vectors into the vector register group based on the first data processing strategy, and uses the single instruction multiple data execution unit to perform odd-even merge sort and merge sort to obtain the top-k data elements in x groups or x1 groups of original data vectors, including: The original data vectors of group x or group x1 are divided into a first sub-vector matrix with y rows and b columns, where y is the data processing capability of a single instruction multiple data execution unit. The insufficient parts in the rows and columns of the first sub-vector matrix are supplemented according to the top-k sorting requirements. The first sub-vector matrix is ​​loaded column-wise into the vector register group, and parity merge sort is performed using the single instruction multiple data execution unit to achieve in-row sorting of the first sub-vector matrix; The first sub-vector matrix after inline sorting is loaded into the vector register group column by column, and the single instruction multiple data execution unit is used to perform one or more rounds of merge sorting to obtain the top-k data elements in the x groups or x1 groups of original data vectors.

3. The data processing apparatus as described in claim 1, characterized in that, The calculation module loads the x21 sets of original data vectors into the vector register group based on the second data processing strategy, and uses the single instruction multiple data execution unit to perform odd-even merge sort and merge sort to obtain the top-k data elements in the x21 sets of original data vectors, including: Divide the m data elements of each group in x21 into n subgroups with granularity b, to obtain a subvector matrix of x21·n rows and b columns; The subvector matrix is ​​divided into a first subvector matrix with y rows and b columns, where y is the data processing capability of a single instruction multiple data execution unit. The insufficient parts in the rows and columns of the first subvector matrix are supplemented according to the top-k sorting requirements. The first sub-vector matrix is ​​loaded column-wise into the vector register group, and parity merge sort is performed using the single instruction multiple data execution unit to achieve in-row sorting of the first sub-vector matrix; The first sub-vector matrix after inline sorting is loaded into the vector register group column by column, and the single instruction multiple data execution unit is used to perform one or more rounds of merge sorting to obtain the top-k data elements in the x21 original data vectors.

4. The data processing apparatus as described in claim 1, characterized in that, The calculation module loads the x22 sets of original data vectors into the vector register group based on the third data processing strategy, and uses the single instruction multiple data execution unit to perform odd-even merge sort and merge sort to obtain the top-k data elements in the x22 sets of original data vectors, including: Perform the following operation on each of the original data vectors in group x22: The original data vector is split based on the target k value to obtain multiple first data vectors; The data elements in each of the first data vectors are mapped to the vector register group, and the single instruction multiple data execution unit is used to perform parallel comparison-exchange operations on the data elements in the vector register group, thereby performing parity merge sort on the data elements in each of the first data vectors to obtain multiple first ordered data vectors. Merge and sort the data elements in the plurality of first ordered data vectors to obtain the top-k data elements in the original data vector.

5. The data processing apparatus according to claim 1, characterized in that, Also includes: An instruction decoding unit, coupled to the storage module, is configured to decode and parse RISC-V instructions extracted from the storage module; The scheduler, coupled to the instruction decoding unit and the computing module respectively, is configured to determine the execution order of each RISC-V instruction based on the instruction characteristics of the RISC-V instructions and the available resources of the computing module, so that the single instruction multiple data execution unit can execute the RISC-V instructions in the order of execution.

6. The data processing apparatus according to claim 5, characterized in that, The storage module includes: A static random access memory, coupled to the computing module, is configured to store data and RISC-V instructions for execution by the single instruction multiple data execution unit.

7. The data processing apparatus according to claim 6, characterized in that, Also includes: An instruction cache unit, coupled to the static random access memory, is configured to retrieve and store RISC-V instructions from the static random access memory. The instruction fetch cache unit, coupled to the instruction cache unit and the instruction decode unit, is configured to store RISC-V instructions fetched from the instruction cache unit in an ordered manner, such that the instruction decode unit fetches RISC-V instructions sequentially from the instruction fetch cache unit.

8. A data processing method, characterized in that, The method includes: Access x sets of original data vectors stored in the storage module, wherein the original data vectors contain m data elements, where m is a positive integer; Determine if x is an integer multiple of the SIMD processing capability; In response to x being an integer multiple of the SIMD processing capability, the x sets of original data vectors are loaded into the vector register group based on the first data processing strategy. Odd-even merge sort and merge sort are performed using a single instruction multiple data execution unit to obtain the top-k data elements in the x sets of original data vectors, where k is a positive integer. In response to x not being an integer multiple of the SIMD processing capability, x is split into x1 and x2, where x1 is an integer multiple of the SIMD processing capability and x2 is the remainder. Based on the first data processing strategy, the x1 group of original data vectors is loaded into the vector register group, and the single instruction multiple data execution unit is used to perform odd-even merge sort and merge sort to obtain the top-k data elements in the x1 group of original data vectors. Determine whether x2 can be split into x21 and x22, where x21×n is an integer multiple of the SIMD processing capability, n is the number of subgroups obtained after splitting m data elements according to b granularity, and x22 is the remaining part; In response to x21 being non-zero, the original data vector of group x21 is loaded into the vector register group based on the second data processing strategy, and the single instruction multiple data execution unit is used to perform odd-even merge sort and merge sort to obtain the top-k data elements in the original data vector of group x21. In response to x22 being non-zero, the x22 original data vectors are loaded into the vector register group based on the third data processing strategy. The single instruction multiple data execution unit is then used to perform odd-even merge sort and merge sort to obtain the top-k data elements in the x22 original data vectors.

9. A chip, characterized in that, Includes the data processing apparatus according to any one of claims 1 to 7.

10. An electronic device, characterized in that, Includes the chip described in claim 9 above.

Images