A data sorting method and device, a terminal and a storage medium

Abstract

The data sorting method, device, terminal and storage medium provided by the application, the method comprises the following steps: determining a composite sorting key of a database in a CPU memory, the composite sorting key comprising a primary sorting key and a secondary sorting key; constructing a key-value pair based on the row pointer of the extracted original data of the database record and the composite sorting key; constructing a target data group suitable for the parallel sorting structure based on the key-value pair; executing the pre-adjusted double-sorting based on the target data group through the GPU; triggering the secondary sorting key comparison when the primary sorting keys of the two data of the to-be-compared data group in the double-sorting sequence constructed based on the target data group are the same in the process of executing the pre-adjusted double-sorting; and assembling the result set in the order of the row pointer of the sorting result through the CPU. When the pre-adjusted double-sorting is executed based on the target data group through the GPU, the comparison logic of the composite sorting key is introduced, so that the secondary sorting key is compared when the primary sorting keys are the same, and the sorting is strictly monotonic.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a data sorting method, device, terminal and storage medium. Background Art

[0002] Currently, when conventional bitonic sorting algorithms are applied in traditional data sorting, they are only based on the comparison of one sort key, ignoring the sorting processing method when the sort keys are the same. As a result, the sorting results cannot meet the first-come-first-served or high-priority first-served requirements in actual scenarios. In addition, in related hardware implementations, such as GPUs, no additional logic is designed to handle sorting when the sort keys are the same, resulting in unstable sorting results for data with the same values, which cannot meet actual sorting requirements.

[0003] Therefore, the existing technology has defects and needs to be improved and developed. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a data sorting method, device, terminal and storage medium in response to the above-mentioned defects of the existing technology, aiming to solve the problems of order conflict and unstable sorting of data with the same value existing in the existing data sorting technology.

[0005] The technical solutions adopted by the present invention to solve the technical problems are as follows:

[0006] In a first aspect, a data sorting method is provided, wherein the method comprises:

[0007] Determine a composite sort key of a database in the CPU memory, and extract row pointers and the composite sort key of multiple original data records in the database; wherein the composite sort key includes a primary sort key and a secondary sort key;

[0008] Constructing corresponding key-value pairs based on the row pointer and the composite sort key, and constructing a target data group adapted to the parallel sorting structure based on the key-value pairs;

[0009] The target data group is loaded into a GPU memory so that the GPU performs a pre-adjusted bitonal sort based on the target data group to obtain a sort result of the key-value pairs; wherein, during the pre-adjusted bitonal sort, when the primary sort keys of two data in the data groups to be compared in the bitonal sequence constructed based on the target data group are the same, a comparison of the secondary sort keys is triggered;

[0010] Determine the row pointer order of the sorting result of the key-value pairs in the GPU display memory so that the CPU assembles the original data according to the row pointer order to obtain a corresponding result set.

[0011] Optionally, before loading the target data group into the GPU memory, the method further includes:

[0012] Determining the amount of data in the target data group, and determining whether the amount of data is a power of 2;

[0013] When the data amount is not a power of 2, preset data is inserted into the target data group until the data amount in the target data group is a power of 2.

[0014] Optionally, the data in the target data group is a numerical value of fixed length.

[0015] Optionally, when the data type of the composite sort key is non-numeric data, before constructing the corresponding key-value pair based on the row pointer and the composite sort key, the method further includes:

[0016] The non-numeric data is converted into a dictionary encoding.

[0017] Optionally, the sub-sort keys in the composite sort key are different from each other.

[0018] Optionally, loading the target data group into a GPU memory so that the GPU performs a pre-adjusted bitonal sort based on the target data group includes:

[0019] Loading the target data set into GPU memory so that the GPU calls a pre-built bitonal sorting system to perform pre-adjusted bitonal sorting based on the target data set;

[0020] The dual-tuning sorting system includes a plurality of sub-comparison systems corresponding to each round of comparison sorting, and the output of the sub-comparison system corresponding to the previous comparison sorting is used as the input of the sub-comparison system corresponding to the next comparison sorting;

[0021] Each of the sub-comparison systems includes a plurality of comparison modules for comparing two data in each to-be-compared data group in the bitonic sequence;

[0022] Each comparison module includes:

[0023] A data comparator, configured to compare the primary sort keys in the composite sort keys corresponding to the two data in the data group to be compared;

[0024] A flag comparator is used to compare the secondary sort keys in the composite sort keys corresponding to the two data when the primary sort keys are the same;

[0025] A sequence correction module is used to receive the first comparison result output by the data comparator and / or the second comparison result output by the flag comparator, correct the first comparison result and / or the second comparison result, and output the sorting result of the two data in the data group to be compared.

[0026] Optionally, the sequence correction module includes:

[0027] a deviation correction unit, configured to receive a first comparison result output by the data comparator indicating that the primary sort keys are identical and a second comparison result output by the flag comparator, and identify and correct a deviation between the first comparison result and the second comparison result;

[0028] A data sequence adjustment or merging unit is used to receive the first comparison result output by the data comparator indicating that the primary sorting keys are different, and based on the corrected comparison result output by the deviation correction unit or the first comparison result indicating that the primary sorting keys are different, adjust or merge the order of the two data in the data group to be compared, and output a corresponding sorting result.

[0029] In a second aspect, the present invention further discloses a data sorting device, wherein the device comprises:

[0030] A sort key determination module is used to determine the composite sort key of the database in the CPU memory;

[0031] A data extraction module, configured to extract row pointers of a plurality of original data records in the database and the composite sort key; wherein the composite sort key includes a primary sort key and a secondary sort key;

[0032] A key-value pair construction module, configured to construct a corresponding key-value pair based on the row pointer and the composite sort key;

[0033] A data group construction module, configured to construct a target data group adapted to a parallel sorting structure based on the key-value pairs;

[0034] a bitonic sorting module, configured to load the target data group into a GPU memory so that the GPU performs a pre-adjusted bitonic sort based on the target data group to obtain a sorting result of the key-value pairs; wherein, during the execution of the pre-adjusted bitonic sorting, when the primary sorting keys of two data in the data groups to be compared in the bitonic sequence constructed based on the target data group are the same, a comparison of the secondary sorting keys is triggered;

[0035] The pointer sequence determination module is used to determine the row pointer sequence of the sorting result of the key-value pairs in the GPU memory, so that the CPU assembles the original data according to the row pointer sequence to obtain a corresponding result set.

[0036] In a third aspect, the present invention discloses a terminal, which includes: a memory, a processor, and a data sorting program stored in the memory and executable on the processor, wherein the data sorting program implements the steps of the data sorting method described above when executed by the processor.

[0037] In a fourth aspect, the present invention discloses a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program can be executed to implement the steps of the data sorting method as described above.

[0038] The present invention provides a data sorting method, device, terminal, and storage medium. The data sorting method includes: determining a composite sort key of a database in a CPU memory, and extracting row pointers and the composite sort key of multiple original data recorded in the database; wherein the composite sort key includes a primary sort key and a secondary sort key; constructing corresponding key-value pairs based on the row pointers and the composite sort key, and constructing a target data group adapted to a parallel sorting structure based on the key-value pairs; loading the target data group into a GPU display memory, so that the GPU performs a pre-adjusted bitonal sort based on the target data group to obtain a sorting result of the key-value pairs; wherein, during the execution of the pre-adjusted bitonal sort, when the primary sort keys of two data groups to be compared in a bitonal sequence constructed based on the target data group are the same, a comparison of the secondary sort keys is triggered; and determining the row pointer order of the sorting result of the key-value pairs in the GPU display memory, so that the CPU assembles the original data according to the row pointer order to obtain a corresponding result set. It can be seen from this that the present invention introduces the comparison logic of the composite sort key when executing the pre-adjusted bitonal sort based on the target data group through the GPU. On the basis of maintaining the efficient parallelism and hardware execution efficiency of the bitonal sort, it can solve the problems of order conflict and unstable sorting of data with the same value existing in the existing data sorting technology, and ensure that when the primary sort keys are the same, the sorting is strictly monotonic by comparing the secondary sort keys. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is a flow chart of a preferred embodiment of the data sorting method of the present invention;

[0040] Figure 2 This is a schematic diagram of a specific bitonal sorting hardware architecture with a sequence correction mechanism disclosed in the present invention;

[0041] Figure 3 It is a specific comparative module schematic diagram disclosed in the present invention;

[0042] Figure 4 This is a schematic diagram of a specific sequence correction module disclosed in the present invention;

[0043] Figure 5 This is a specific bitonal sorting schematic diagram disclosed in the present invention;

[0044] Figure 6 This is a functional principle block diagram of a preferred embodiment of the data sorting device of the present invention;

[0045] Figure 7 It is a functional principle block diagram of a preferred embodiment of the terminal in the present invention. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solutions and advantages of the present invention more clear and distinct, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0047] See Figure 1 , Figure 1 This is a flow chart of the data sorting method in the present invention. Figure 1 As shown, the data sorting method according to the embodiment of the present invention includes:

[0048] Step S11, determining the composite sort key of the database in the CPU memory, and extracting row pointers of multiple original data recorded in the database and the composite sort key; wherein the composite sort key includes a primary sort key and a secondary sort key.

[0049] In this embodiment, in order to overcome the problem that the sorting result may be unstable when the sorting key values ​​are the same when relying solely on a single sorting key for sorting, a composite sorting key is introduced. That is, when sorting the data stored in the CPU memory database, the composite sorting key value of the database in the CPU memory is first determined so that the data therein can be sorted based on the composite sorting key later, that is, the primary sorting key and the secondary sorting key of the database in the CPU memory are determined, and the primary sorting key and the secondary sorting key constitute the composite sorting key. It should be noted that the secondary sorting keys in the composite sorting key are different from each other, that is, the key values ​​corresponding to the secondary sorting keys are different, and the database supports GPU acceleration, and the amount of data in the database is also extremely large, such as hundreds of millions of rows. For example, when the original data of the database in the CPU memory is as shown in Table 1, the grades are determined as the primary sorting key and the student ID is determined as the secondary sorting key, that is, the grades and the student ID constitute the composite sorting key.

[0050] Table 1

[0051]

[0052] In this embodiment, after determining the composite sort key used for database sorting, the row pointers and composite sort keys of multiple original data recorded in the database are extracted, so that only the row pointers and composite sort keys are subsequently loaded into the GPU video memory, thereby minimizing data transmission and reducing the time overhead of data handling.

[0053] Step S12: construct corresponding key-value pairs based on the row pointer and the composite sort key, and construct a target data group adapted to the parallel sorting structure based on the key-value pairs.

[0054] In this embodiment, after extracting row pointers and composite sort keys from multiple pieces of raw data recorded in the database, corresponding key-value pairs are constructed based on the row pointers and composite sort keys, such as (0x1000, (85, S001)), (0x1040, (92, S002)), (0x1080, (78, S003)), (0x1120, (92, S004)), etc. Furthermore, in this embodiment, when the data type of the composite sort key is non-numeric data, before constructing the corresponding key-value pair based on the row pointer and composite sort key, the process may further specifically include: converting the non-numeric data into a dictionary code. It is understandable that parallel hardware requires deterministic memory access, so when the data type of the composite sort key is non-numeric data, converting the non-numeric data into a dictionary code can ensure that the length of the composite sort key is fixed. Specifically, a dictionary corresponding to the composite sort key is created, and the composite sort key is converted into numeric data based on the dictionary.

[0055] In this embodiment, corresponding key-value pairs are constructed based on row pointers and composite sort keys. A target data group adapted to the parallel sorting structure can be constructed based on these key-value pairs, for example, [(0x1000, (85, S001)), (0x1040, (92, S002)), (0x1080, (78, S003)), (0x1120, (92, S004)), ...]. The data in the target data group is a fixed-length value.

[0056] Step S13: Load the target data group into the GPU memory so that the GPU performs a pre-adjusted bitonal sort based on the target data group to obtain a sorting result of the key-value pairs; wherein, during the process of performing the pre-adjusted bitonal sort, when the primary sorting keys of two data in the data groups to be compared in the bitonal sequence constructed based on the target data group are the same, comparison of the secondary sorting keys is triggered.

[0057] It is understandable that since the GPU cannot directly access the CPU data, it is necessary to load the CPU data into the GPU memory, that is, the target data group based on the adaptive parallel sorting structure constructed based on the key-value pairs is loaded into the GPU memory, so that the GPU can perform the pre-adjusted bitonal sorting based on the target data group. Among them, the bitonal sorting has high parallelism on the GPU and high memory bandwidth utilization.

[0058] It should be noted that the pre-adjusted bitonal sort is different from the conventional bitonal sort. The conventional bitonal sort is based only on element value sorting and lacks processing for the priority of elements with the same value. However, in this embodiment, during the execution of the pre-adjusted bitonal sort, when the primary sort keys of two data in the data group to be compared in the bitonal sequence constructed based on the target data group are the same, the comparison of the secondary sort keys is triggered. That is, the overall sorting logic of the pre-adjusted bitonal sort is still the same as that of the conventional bitonal sort. However, when comparing data, if the primary sort keys of the data are the same, the secondary sort keys are compared.

[0059] In this embodiment, before loading the target data group into the GPU memory, the following steps may be specifically performed: determining the amount of data in the target data group and determining whether the amount of data is a power of 2; if the amount of data is not a power of 2, inserting preset data into the target data group until the amount of data in the target data group is a power of 2. After the sorting is completed, the preset data is removed from the sorting result. It is understandable that in bitonic sorting, the number of elements in the bitonic sequence needs to be 2. n , where n is a positive integer. Therefore, before loading the target data group into the GPU memory and performing the pre-adjusted bitonal sorting based on the target data group by the GPU, it is necessary to ensure that the amount of data in the target data group is a power of 2.

[0060] In this embodiment, the target data group is loaded into the GPU memory so that the GPU performs a pre-adjusted bitonal sort based on the target data group. Specifically, the target data group is loaded into the GPU memory so that the GPU calls a pre-built bitonal sorting system to perform the pre-adjusted bitonal sort based on the target data group. The bitonal sorting system includes a plurality of sub-comparison systems corresponding to each comparison and sorting round, and the output of the sub-comparison system corresponding to the previous comparison and sorting round serves as the input of the sub-comparison system corresponding to the next comparison and sorting round. Each sub-comparison system includes a plurality of comparison modules for comparing two data in each data group to be compared in the bitonal sequence. Each comparison module includes: a data comparator for comparing the primary sorting keys in the composite sorting keys corresponding to the two data in the data group to be compared; a flag comparator for comparing the secondary sorting keys in the composite sorting keys corresponding to the two data when the primary sorting keys are the same; and a sequence correction module for receiving a first comparison result output by the data comparator and / or a second comparison result output by the flag comparator, correcting the first comparison result and / or the second comparison result, and outputting a sorting result for the two data in the data group to be compared.

[0061] Furthermore, the sequence correction module may further specifically include:

[0062] a deviation correction unit, configured to receive a first comparison result output by the data comparator indicating that the primary sort keys are identical and a second comparison result output by the flag comparator, and identify and correct a deviation between the first comparison result and the second comparison result;

[0063] The data sequence adjustment or merging unit is used to receive the first comparison result output by the data comparator indicating that the primary sorting keys are different, and based on the corrected comparison result output by the deviation correction unit or the first comparison result indicating that the primary sorting keys are different, adjust or merge the order of the two data in the compared data group and output the corresponding sorting result.

[0064] For example, see Figure 2 As shown, 8 groups of input data are efficiently processed in this dual-tone sorting system through time-division multiplexing technology. In different clock cycles, different data groups will be sent to the comparison module for comparison and sorting.

[0065] See also Figure 3 As shown, each comparison module is equipped with a data comparator and a flag comparator. The data comparator is responsible for comparing the size of the input data, while the flag comparator takes effect when the data is equal. That is, when the result output by the data comparator is equal, the flag comparator is enabled to compare the secondary sorting key corresponding to the data. These results are then input into the sequence correction module to ensure that when the data is equal, the sorting can still be correctly corrected according to the flag signal.

[0066] See also Figure 4 As shown, the input data Data_cmp_eql and the condition flag Flag_gt enter the deviation correction unit. The deviation correction unit is responsible for identifying and adjusting the deviation in the data to ensure the accuracy of the data. The corrected data enters the data sequence adjustment or merging module to adjust or merge the order of the data to meet the needs of subsequent processing and ensure the consistency and orderliness of the data. Finally, the processed data is output in the form of [flag, data]. These data have been corrected and adjusted to ensure the accuracy and consistency of the output data.

[0067] Data_cmp_eq indicates that the data comparator's comparison result is equal, requiring the subsequent deviation correction module. Data_cmp_gt indicates that the data comparator's comparison result is greater than, and Data_cmp_less indicates that the data comparator's comparison result is less than. The comparison results corresponding to these two cases do not need to be input into the deviation correction module. Flag_gt represents the output of the flag comparator. This signal serves as the correction condition for the sequence correction module to perform subsequent correction operations. The sequence correction module performs necessary corrections to the data based on the comparison results within each clock cycle. The entire sorting process can be completed within 5 clock cycles, ensuring a strictly monotonically increasing sorting output.

[0068] For example, the global bitonic sequence [1 1 , 7 2 , 8 0 , 9 3 , 8 7 , 7 5 , 6 4 , 6 6 ]When sorting in ascending order, multiple groups of elements to be compared [1 1 , 8 7 ]、[7 2 , 7 5 ]、[8 0 , 6 4 ]、[9 3 , 6 6 ], parallel comparison, that is, executing 1 at the same time 1 and 8 7 Comparison, 7 2 and 7 5 Comparison, 8 0 and 6 4 Comparison, 9 3 and 6 6 The comparison of 1 and 8 7 If the values ​​of two elements are different, the sorting positions of the two elements are swapped based on the value, such as sorting in ascending order of value. So for [1 1 , 8 7 ] This group of elements to be compared does not need to be exchanged 1 1 and 8 7 The sort position, 7 2 and 7 5 Compare, if the values ​​of two elements are the same, swap the sorting positions of the two elements based on priority, such as sorting in descending order of priority, that is, swap 7 2 and 7 5 The position of , similarly, 8 0 and 6 4Compare and sort in ascending order by value, that is, swap 8 0 and 6 4 The sort position of 9 3 and 6 6 Compare and sort in ascending order by value, that is, swap 9 3 and 6 6 The sorting position of the current comparison and exchange round is [1 1 , 7 5 , 6 4 , 6 6 , 8 7 , 7 2 , 8 0 , 9 3 ].

[0069] Moreover, after each round of comparison and exchange is completed, it is determined whether the current sorted sequence is an ordered sequence; when the current sorted sequence is not an ordered sequence, multiple new groups of elements to be compared in the current sorted sequence are determined, and the two elements in each new group of elements to be compared are compared using the composite key values ​​corresponding to the elements to obtain the corresponding comparison results until the current sorted sequence is an ordered sequence. After the above steps, the final ordered sequence is [1 1 , 6 6 , 6 4 , 7 5 , 7 2 , 8 7 , 8 0 , 9 3 ].

[0070] For example, see Figure 5 As shown, the target data set is [8 0 , 1 1 , 7 2 , 9 3 , 6 4 , 7 5 , 6 6 , 8 7 ] is sorted, it can be seen that the sorting result obtained by the conventional double-tuning sorting method is [1 1 , 6 6 , 6 4 , 7 2 , 7 5 , 8 7 , 8 0 , 9 3 ], the conventional two-tone sorting method does not introduce priority sorting, which leads to unstable order of elements with the same value. However, the sorting result obtained by the two-tone sorting method pre-adjusted by this application is [1 1 , 6 6, 6 4 , 7 5 , 7 2 , 8 7 , 8 0 , 9 3 ], on the basis of retaining the parallelism of bitonic sorting, the priority is taken into consideration to ensure that elements with the same numerical value are sorted according to the priority, so that the final sorting result meets the sorting requirements, solves the order conflict problem of elements with the same numerical value, and meets the needs of high-precision scenarios such as real-time scheduling and financial transactions.

[0071] Step S14: determining the row pointer order of the sorting result of the key-value pairs in the GPU display memory so that the CPU assembles the original data according to the row pointer order to obtain a corresponding result set.

[0072] In this embodiment, after the GPU outputs the sorting result, the row pointer sequence of the sorting result can be determined, and the CPU can assemble the corresponding result set according to the row pointer sequence.

[0073] For example, after the target data [(0x1000, (85, S001)), (0x1040, (92, S002)), (0x1080, (78, S003)), (0x1120, (92, S004))] is sorted in advance, the sorting result is [(0x1040, (92, S002)), (0x1120, (92, S004)), (0x1000, (85, S001)), (0x1080, (78, S003))], and the row pointer order is (0x1040, 0x1120, 0x1000, 0x1080). The CPU assembles the corresponding result set according to the row pointer order as shown in Table 2.

[0074] Table 2

[0075]

[0076] It can be seen that in the embodiment of the present invention, when the pre-adjusted bitonal sort is performed based on the target data group by the GPU, the comparison logic of the composite sort key is introduced. This can solve the problems of order conflicts and unstable sorting of data with the same value existing in the existing data sorting technology on the basis of maintaining the efficient parallelism and hardware execution efficiency of the bitonal sort, and ensure that when the primary sort keys are the same, the sorting is strictly monotonic by comparing the secondary sort keys.

[0077] In one embodiment, if Figure 6 As shown, based on the above data sorting method, the present invention also provides a data sorting device, including:

[0078] A sort key determination module 11 is used to determine the composite sort key of the database in the CPU memory;

[0079] A data extraction module 12 is configured to extract row pointers and the composite sort key of the plurality of original data records in the database; wherein the composite sort key includes a primary sort key and a secondary sort key;

[0080] A key-value pair construction module 13 is configured to construct a corresponding key-value pair based on the row pointer and the composite sort key;

[0081] A data group construction module 14 is configured to construct a target data group adapted to a parallel sorting structure based on the key-value pairs;

[0082] The bitonic sorting module 15 is configured to load the target data group into the GPU memory so that the GPU performs a pre-adjusted bitonic sort based on the target data group to obtain a sorting result of the key-value pairs; wherein, during the pre-adjusted bitonic sorting, when the primary sorting keys of two data in the data groups to be compared in the bitonic sequence constructed based on the target data group are the same, a comparison of the secondary sorting keys is triggered;

[0083] The pointer sequence determining module 16 is used to determine the row pointer sequence of the sorting result of the key-value pairs in the GPU memory, so that the CPU assembles the original data according to the row pointer sequence to obtain a corresponding result set.

[0084] In some specific embodiments, the data sorting device may further include:

[0085] a data volume determination module, configured to determine the data volume in the target data group and determine whether the data volume is a power of 2;

[0086] A data insertion module is used to insert preset data into the target data group when the data amount is not a power of 2, until the data amount in the target data group is a power of 2.

[0087] In some specific embodiments, the data sorting device may further include:

[0088] The data type conversion module is used to convert the non-numeric data into a dictionary code.

[0089] In some specific embodiments, the bitune sorting module 15 may specifically include:

[0090] A sorting unit, configured to load the target data group into a GPU memory so that the GPU calls a pre-built bitonal sorting system to perform a pre-adjusted bitonal sort based on the target data group;

[0091] The dual-tuning sorting system includes a plurality of sub-comparison systems corresponding to each round of comparison sorting, and the output of the sub-comparison system corresponding to the previous comparison sorting is used as the input of the sub-comparison system corresponding to the next comparison sorting;

[0092] Each of the sub-comparison systems includes a plurality of comparison modules for comparing two data in each to-be-compared data group in the bitonic sequence;

[0093] Each comparison module includes:

[0094] A data comparator, configured to compare the primary sort keys in the composite sort keys corresponding to the two data in the data group to be compared;

[0095] A flag comparator is used to compare the secondary sort keys in the composite sort keys corresponding to the two data when the primary sort keys are the same;

[0096] A sequence correction module is used to receive the first comparison result output by the data comparator and / or the second comparison result output by the flag comparator, correct the first comparison result and / or the second comparison result, and output the sorting result of the two data in the data group to be compared.

[0097] In some specific embodiments, the sequence correction module may specifically include:

[0098] a deviation correction unit, configured to receive a first comparison result output by the data comparator indicating that the primary sort keys are identical and a second comparison result output by the flag comparator, and identify and correct a deviation between the first comparison result and the second comparison result;

[0099] A data sequence adjustment or merging unit is used to receive the first comparison result output by the data comparator indicating that the primary sorting keys are different, and based on the corrected comparison result output by the deviation correction unit or the first comparison result indicating that the primary sorting keys are different, adjust or merge the order of the two data in the data group to be compared, and output a corresponding sorting result.

[0100] In addition, it is worth noting that the working process of a data sorting device provided in this embodiment is the same as the working process of the above-mentioned data sorting method, which will not be described in detail here. For details, please refer to the working process of the above-mentioned data sorting method.

[0101] Figure 7 This is a schematic diagram of the structure of a terminal provided in an embodiment of the present application. The terminal may include:

[0102] Memory 501 , processor 502 , and computer programs stored in the memory 501 and executable on the processor 502 .

[0103] When the processor 502 executes the program, the data sorting method provided in the above embodiment is implemented.

[0104] Furthermore, the terminal further includes:

[0105] The communication interface 503 is used for communication between the memory 501 and the processor 502 .

[0106] The memory 501 is used to store computer programs that can be run on the processor 502 .

[0107] The memory 501 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0108] If memory 501, processor 502, and communication interface 503 are implemented independently, communication interface 503, memory 501, and processor 502 can be interconnected via a bus to facilitate communication. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. Buses can be categorized as address buses, data buses, and control buses. For ease of illustration, the figure uses only one line, but this does not imply that there is only one bus or only one type of bus.

[0109] Optionally, in a specific implementation, if the memory 501, the processor 502 and the communication interface 503 are integrated on a chip, the memory 501, the processor 502 and the communication interface 503 can communicate with each other through an internal interface.

[0110] The processor 502 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0111] This embodiment also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the above data sorting method is implemented.

[0112] Other embodiments of the present invention will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses, or adaptations of the present invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the invention being indicated by the claims.

[0113] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example" or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0114] The logic and / or steps represented in the flowchart or otherwise described herein may be considered, for example, as a sequenced list of executable instructions for implementing the logical functions, and may be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other system that can read and execute instructions from an instruction execution system, apparatus, or device).

[0115] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiment, the N steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. If implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used: a discrete logic circuit having logic gate circuits for implementing logical functions on data signals, an application-specific integrated circuit having suitable combinational logic gate circuits, a programmable gate array (PGA), a field-programmable gate array (FPGA), etc.

[0116] It should be understood that the application of the present invention is not limited to the above examples. For those skilled in the art, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. A data sorting method, characterized in that: The method comprises: Determine a composite sort key of a database in the CPU memory, and extract row pointers and the composite sort key of multiple original data records in the database; wherein the composite sort key includes a primary sort key and a secondary sort key; Constructing corresponding key-value pairs based on the row pointer and the composite sort key, and constructing a target data group adapted to the parallel sorting structure based on the key-value pairs; The target data group is loaded into a GPU memory so that the GPU performs a pre-adjusted bitonal sort based on the target data group to obtain a sort result of the key-value pairs; wherein, during the pre-adjusted bitonal sort, when the primary sort keys of two data in the data groups to be compared in the bitonal sequence constructed based on the target data group are the same, a comparison of the secondary sort keys is triggered; Determining a row pointer order of the sorted result of the key-value pairs in the GPU memory, so that the CPU assembles the original data according to the row pointer order to obtain a corresponding result set; The step of loading the target data group into a GPU memory so that the GPU performs a pre-adjusted bitonal sort based on the target data group includes: Loading the target data set into GPU memory so that the GPU calls a pre-built bitonal sorting system to perform pre-adjusted bitonal sorting based on the target data set; The dual-tuning sorting system includes a plurality of sub-comparison systems corresponding to each round of comparison sorting, and the output of the sub-comparison system corresponding to the previous comparison sorting is used as the input of the sub-comparison system corresponding to the next comparison sorting; Each of the sub-comparison systems includes a plurality of comparison modules for comparing two data in each to-be-compared data group in the bitonic sequence; Each comparison module includes: A data comparator, configured to compare the primary sort keys in the composite sort keys corresponding to the two data in the data group to be compared; A flag comparator is used to compare the secondary sort keys in the composite sort keys corresponding to the two data when the primary sort keys are the same; A sequence correction module is used to receive the first comparison result output by the data comparator and / or the second comparison result output by the flag comparator, correct the first comparison result and / or the second comparison result, and output the sorting result of the two data in the data group to be compared.

2. The data sorting method according to claim 1, characterized in that: Before loading the target data group into the GPU memory, the method further includes: Determining the amount of data in the target data group, and determining whether the amount of data is a power of 2; When the data amount is not a power of 2, preset data is inserted into the target data group until the data amount in the target data group is a power of 2.

3. The data sorting method according to claim 1, wherein: The data in the target data group are numerical values ​​of fixed length.

4. The data sorting method according to claim 1, wherein: When the data type of the composite sort key is non-numeric data, before constructing the corresponding key-value pair based on the row pointer and the composite sort key, the method further includes: The non-numeric data is converted into a dictionary encoding.

5. The data sorting method according to claim 1, wherein: The sub-sort keys in the composite sort key are different from each other.

6. The data sorting method according to claim 1, wherein: The sequence correction module includes: a deviation correction unit, configured to receive a first comparison result output by the data comparator indicating that the primary sort keys are identical and a second comparison result output by the flag comparator, and identify and correct a deviation between the first comparison result and the second comparison result; A data sequence adjustment or merging unit is used to receive the first comparison result output by the data comparator indicating that the primary sorting keys are different, and based on the corrected comparison result output by the deviation correction unit or the first comparison result indicating that the primary sorting keys are different, adjust or merge the order of the two data in the data group to be compared, and output a corresponding sorting result.

7. A data sorting device, characterized in that: The device comprises: A sort key determination module is used to determine the composite sort key of the database in the CPU memory; A data extraction module, configured to extract row pointers and the composite sort key from a plurality of original data records in a database; wherein the composite sort key comprises a primary sort key and a secondary sort key; A key-value pair construction module, configured to construct a corresponding key-value pair based on the row pointer and the composite sort key; A data group construction module, configured to construct a target data group adapted to a parallel sorting structure based on the key-value pairs; a bitonic sorting module, configured to load the target data group into a GPU memory so that the GPU performs a pre-adjusted bitonic sort based on the target data group to obtain a sorting result of the key-value pairs; wherein, during the execution of the pre-adjusted bitonic sorting, when the primary sorting keys of two data in the data groups to be compared in the bitonic sequence constructed based on the target data group are the same, a comparison of the secondary sorting keys is triggered; a pointer order determination module, configured to determine a row pointer order of the sorted result of the key-value pairs in the GPU memory, so that the CPU assembles the original data according to the row pointer order to obtain a corresponding result set; Wherein, the bitune sorting module includes: A sorting unit, configured to load the target data group into a GPU memory so that the GPU calls a pre-built bitonal sorting system to perform a pre-adjusted bitonal sort based on the target data group; The dual-tuning sorting system includes a plurality of sub-comparison systems corresponding to each round of comparison sorting, and the output of the sub-comparison system corresponding to the previous comparison sorting is used as the input of the sub-comparison system corresponding to the next comparison sorting; Each of the sub-comparison systems includes a plurality of comparison modules for comparing two data in each to-be-compared data group in the bitonic sequence; Each comparison module includes: A data comparator, configured to compare the primary sort keys in the composite sort keys corresponding to the two data in the data group to be compared; A flag comparator is used to compare the secondary sort keys in the composite sort keys corresponding to the two data when the primary sort keys are the same; A sequence correction module is used to receive the first comparison result output by the data comparator and / or the second comparison result output by the flag comparator, correct the first comparison result and / or the second comparison result, and output the sorting result of the two data in the data group to be compared.

8. A terminal, characterized in that: include: A memory, a processor, and a data sorting program stored in the memory and executable on the processor, wherein the data sorting program, when executed by the processor, implements the steps of the data sorting method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which can be executed to implement the steps of the data sorting method according to any one of claims 1 to 6.

Images