Data sorting method and apparatus
By separating and storing the sort key and additional fields and sorting them independently, the problem of excessive cache space consumption during the sorting process is solved, improving sorting efficiency and cache hit rate.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIJING OCEANBASE TECHNOLOGY CO LTD
- Filing Date
- 2023-12-15
- Publication Date
- 2026-07-10
AI Technical Summary
Existing technologies require storing both the sort key and additional fields when sorting data, resulting in excessive cache space usage and low sorting efficiency.
The sort key and additional fields are stored separately in memory. The sort key data is sorted according to a preset sorting rule and then merged to obtain the sorted result.
By separating the data to be sorted and the additional data, the amount of data accessed during the sorting process is reduced, sorting efficiency is improved, and cache hit rate is increased.
Smart Images
Figure CN117763008B_ABST
Abstract
Description
Technical Field
[0001] This specification relates to the field of database technology, and more particularly to a data sorting method and apparatus. Background Technology
[0002] Data sorting refers to the operation of arranging data stored in a database according to specific rules. Related technologies typically store the data to be sorted (using the sort key) and the data not used for sorting (using additional fields) together during the sorting process. Then, the data in the sort key and the data in the additional fields are treated as a whole and sorted accordingly. This approach consumes additional cache space, resulting in low sorting efficiency. Summary of the Invention
[0003] In view of the above, one or more embodiments of this specification provide a data sorting method and apparatus.
[0004] To achieve the above objectives, one or more embodiments of this specification provide the following technical solutions:
[0005] According to a first aspect of one or more embodiments of this specification, a data sorting method is proposed, comprising:
[0006] In the data table, determine the sort key and the additional fields associated with the sort key. The sort key is used to store at least one row of data to be sorted, and the additional fields are used to store additional data corresponding to each row of data to be sorted.
[0007] Store at least one row of data to be sorted and the corresponding additional data separately in memory;
[0008] Sort at least one row of data to be sorted stored in memory according to a preset sorting rule;
[0009] The additional data stored in memory is merged with at least one row of the sorted data to obtain the sorted result.
[0010] According to a second aspect of one or more embodiments of this specification, a data sorting apparatus is provided, comprising:
[0011] The determination module is used to determine the sort key and the additional fields associated with the sort key in the data table. The sort key is used to store at least one row of data to be sorted, and the additional fields are used to store additional data corresponding to each row of data to be sorted.
[0012] The storage module is used to independently store at least one row of data to be sorted and the corresponding additional data into memory;
[0013] The sorting module is used to sort at least one row of data to be sorted stored in memory according to a preset sorting rule;
[0014] The merge module is used to merge the additional data stored in memory with at least one row of the sorted data to be sorted, and obtain the sorted result.
[0015] According to a third aspect of one or more embodiments of this specification, an electronic device is provided, comprising:
[0016] processor;
[0017] Memory used to store processor-executable instructions;
[0018] The processor implements the method described in the first aspect by running executable instructions.
[0019] According to a fourth aspect of one or more embodiments of this specification, a computer-readable storage medium is provided that stores computer instructions thereon, which, when executed by a processor, implement the steps of the method as described in the first aspect.
[0020] The solution provided in this specification allows for the determination of a sort key and associated additional fields in a data table. The data to be sorted, stored in the sort key, and the additional data stored in the additional fields are then independently stored in memory. Following a preset sorting rule, at least one row of data to be sorted in memory is sorted, and the additional data in memory is merged with the sorted at least one row of data to be sorted to obtain the sorted result. By storing the data to be sorted and the additional data separately, this embodiment allows for the individual retrieval of the data to be sorted during the sorting process. This reduces the amount of data that needs to be retrieved during sorting and improves sorting efficiency. Attached Figure Description
[0021] Figure 1 This is a flowchart illustrating a data sorting method provided in an exemplary embodiment.
[0022] Figure 2 This is a schematic diagram illustrating the specific process of a data sorting method provided in an exemplary embodiment.
[0023] Figure 3 This is a schematic diagram of the structure of a device provided in an exemplary embodiment.
[0024] Figure 4 This is a schematic diagram of the structure of a data sorting device provided in an exemplary embodiment. Detailed Implementation
[0025] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with one or more embodiments of this specification. Rather, they are merely examples of apparatuses and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.
[0026] It should be noted that the steps of the corresponding methods are not necessarily performed in the order shown and described in this specification in other embodiments. In some other embodiments, the methods may include more or fewer steps than described in this specification. Furthermore, a single step described in this specification may be broken down into multiple steps in other embodiments; and multiple steps described in this specification may be combined into a single step in other embodiments.
[0027] Data sorting refers to the operation of arranging data stored in a database according to specific rules. Related technologies typically store the data in the sort key that is to be sorted along with the data in the additional fields that are not to be sorted. During the sorting process, the data in the sort key and the data in the additional fields are treated as a whole and then rearranged and sorted.
[0028] However, in most cases, the sort key might be an integer or date type, requiring very little storage space. The additional fields, on the other hand, might contain a large amount of data, requiring a significant amount of storage space.
[0029] According to the sorting methods provided by relevant technologies, storing the sorting key and additional fields together may cause the stored data to greatly exceed the capacity of the CPU cache, resulting in a low cache hit rate during the sorting process. Each key-value comparison requires accessing main memory, leading to poor performance and low sorting efficiency.
[0030] In view of this, the solution provided in this specification separates the smaller sort key data from the larger additional field data, so that the sort key data can be accessed separately during the sorting process. This allows as much sort key data as possible to be stored in the CPU cache, improving the cache hit rate and thus improving sorting efficiency.
[0031] Specifically, the solution provided in this specification allows for the determination of a sort key and associated additional fields in a data table. The data to be sorted stored in the sort key and the additional data stored in the additional fields are then independently stored in memory. Following a preset sorting rule, at least one row of data to be sorted in memory is sorted, and the additional data in memory is merged with the sorted at least one row of data to be sorted to obtain the sorted result. By storing the data to be sorted and the additional data separately, this embodiment allows for the individual retrieval of the data to be sorted during the sorting process. This reduces the amount of data that needs to be retrieved during sorting and improves sorting efficiency.
[0032] The exemplary embodiments of this specification will now be described in detail.
[0033] First, this specification provides a data sorting method that can be executed by any electronic device.
[0034] Figure 1 This is a flowchart illustrating a data sorting method provided in an exemplary embodiment, such as... Figure 1 As shown in the embodiments of this specification, the data sorting method includes the following steps.
[0035] S101, determine the sort key and the additional fields associated with the sort key in the data table. The sort key is used to store at least one row of data to be sorted, and the additional fields are used to store additional data corresponding to each row of data to be sorted.
[0036] It's important to note that the data table can be any one or more tables in the database, serving as the query scope for Structured Query Language (SQL) statements. The sort key and additional fields can be fields from the SQL query results. The sort key is the field directly involved in sorting the results, while additional fields are fields that are not involved in sorting. For example, in the SQL statement "select c1,c2,c3 from t orderby c1", "t" is the data table, and "c1", "c2", and "c3" are three fields in the data table. "c1" is determined as the sort key, and "c2" and "c3" are determined as additional fields. By determining at least some fields as the sort key, the SQL query results can be output according to the order of the data to be sorted in the sort key (e.g., sorted in ascending numerical order, or alphabetically from A to Z).
[0037] It's understandable that there can be one or more sort keys. When there are multiple sort keys, the data to be sorted is first compared to the data under one of the sort keys. If the data under that sort key is the same, then the data under other sort keys is compared. For example, in the SQL statement "select c1,c2,c3 from t order by c1,c2", there are two fields, "c1" and "c2", which are determined as sort keys. During the sorting process, the data under "c1" is first compared to the data under "c2". If the data under "c1" is the same, then the data under "c2" is compared.
[0038] In other words, when there are multiple sort keys, the data to be sorted in each sort key may participate in the sorting process, and the final sorting result depends on the data to be sorted in multiple sort keys.
[0039] Furthermore, it's understandable that there can be one or more additional fields. Since the additional data in these fields doesn't directly participate in the sorting process, but is output according to the sorted result of the data to be sorted, regardless of the number of additional fields, the key is to ensure that the additional data in these fields corresponds to the data to be sorted in the sort key. The additional data in the sorted result can then be displayed alongside the sorted data according to their correspondence with the data to be sorted.
[0040] S102, store at least one row of data to be sorted and the corresponding additional data separately into memory.
[0041] It should be noted that the data to be sorted and the additional data can be encoded into data types that can be used for sorting, and materialized and stored in memory so that the CPU can call the data to be sorted and the additional data to complete the sorting operation.
[0042] By storing the data to be sorted and the supplementary data separately, the sorting process can focus on reading and sorting only the data to be sorted, and then merging the supplementary data with the sorted data to be sorted after the sorting is complete, thereby improving sorting efficiency.
[0043] Since there is a correspondence between the data to be sorted and the supplementary data, it is necessary to maintain the correspondence between the two during the process of storing them separately. This is so that after sorting at least one row of data to be sorted independently, the supplementary data corresponding to each row of data to be sorted can be merged according to the correspondence to obtain the completed sorting result.
[0044] In some embodiments, an identifier field can be added to both the separately stored sorted data and supplementary data to indicate the correspondence between them. For example, the same identifier can be added to corresponding sorted data and supplementary data using the identifier field.
[0045] In some embodiments, additional data can be redundantly stored in each row of unsorted data, or additional data can be redundantly stored in each row of additional data, so that the correspondence between the two can be obtained based on the redundantly stored data.
[0046] In some embodiments, at least one row of data to be sorted and corresponding supplementary data can be stored in memory in a row-oriented manner. Each row of data to be sorted in the row-oriented storage includes an supplementary pointer field, which stores the memory address of the supplementary data corresponding to each row of data to be sorted. It can be understood that the supplementary pointer field records pointers to each row of data to be sorted; since the supplementary data is stored in memory in a row-oriented manner, these pointers can directly point to the memory address of the data row containing the supplementary data.
[0047] By adding an additional pointer field to the data to be sorted, a correspondence between the data to be sorted and the additional data can be established without introducing extra data or redundant data storage, greatly saving memory usage. In scenarios with large amounts of data, it can increase the amount of data stored in memory each time, thereby improving sorting speed.
[0048] S103, sort at least one row of data to be sorted stored in memory according to a preset sorting rule.
[0049] It should be noted that the preset sorting rule can be a user-specified sorting rule. For example, for numeric data to be sorted, it can be sorted in ascending or descending order according to the numerical value. For string data to be sorted, it can be sorted in ascending or descending order according to the ASCII value of the first character. This specification does not limit this.
[0050] In some embodiments, the data to be sorted can be called independently for sorting without simultaneously calling additional data during the sorting process. Since the size of the data to be sorted is usually much smaller than the size of the additional data, the entire data to be sorted can be stored in the CPU cache to further improve sorting efficiency.
[0051] For example, at least one row of data to be sorted can be stored in memory and then placed into the CPU cache. Subsequently, the at least one row of data to be sorted in the CPU cache is sorted according to a preset sorting rule. After the at least one row of data to be sorted in the CPU cache is sorted, the sorted at least one row of data to be sorted is returned to memory so that it can be merged with additional data in memory to obtain a complete sorting result.
[0052] It is understandable that, since the access speed of the CPU cache is much higher than that of the memory, storing the data to be sorted in the CPU cache can greatly improve sorting efficiency, as all the data that needs to be accessed during the sorting process can be hit in the CPU cache.
[0053] S104: Merge the additional data stored in memory with at least one row of data to be sorted after sorting to obtain the sorting result.
[0054] In some embodiments, when an additional pointer field is added to each row of unsorted data stored separately, a corresponding additional pointer field also exists in each row of unsorted data after sorting. By replacing the additional pointer field in at least one row of unsorted data after sorting with the corresponding additional data stored in memory, the unsorted data and the additional data can be merged to output a complete sorting result.
[0055] Therefore, the solution provided in this manual improves sorting efficiency by storing the unsorted data in the sort key and the supplementary data in the supplementary field separately, allowing the unsorted data to be retrieved individually during the sorting process. After sorting the unsorted data, each row of supplementary data is then merged with the sorted unsorted data, ultimately outputting the complete sorting result.
[0056] In some embodiments, the process of separating and merging the data to be sorted and the additional data also incurs a certain amount of operation time. In some application scenarios (e.g., when the additional data is small), this operation time may make the above-mentioned solution of separating and sorting the data not the optimal solution.
[0057] To address the aforementioned issues, the embodiments in this specification can also adaptively select an appropriate sorting method based on the actual application scenario, thereby automatically selecting a better sorting method in different application scenarios. For specific implementation details, please refer to the following description.
[0058] For example, the size of at least one row of data to be sorted can be predicted before storing the data to be sorted and the additional data separately. If the size of the data to be sorted is smaller than the size of the CPU cache, at least one row of data to be sorted and the corresponding additional data are then stored separately in memory, and the method steps shown in S103 to S104 above are then executed.
[0059] Since the size of the data to be sorted is smaller than the CPU cache size, it can be guaranteed that the data to be sorted can be stored in the CPU cache all at once, thus avoiding repeated memory operations during the sorting process. In this case, the sorting cost of storing the data to be sorted and the accompanying data separately and then sorting them is relatively low, so this sorting method can be chosen.
[0060] It should be noted that the CPU cache mentioned in this manual can refer to the CPU's Level 3 cache (L3 cache). The L3 cache is the largest cache in the CPU, which can accommodate as much data as possible to be sorted, reduce memory operations during the sorting process, and thus improve sorting efficiency.
[0061] For example, before storing the data to be sorted and the additional data separately, the sorting cost of storing at least one row of data to be sorted and the corresponding additional data separately in memory, as well as the sorting cost of storing at least one row of data to be sorted together with the corresponding additional data in memory, can be predicted. If the sorting cost of storing them separately in memory is less than the sorting cost of storing them together in memory, then at least one row of data to be sorted and the corresponding additional data can be stored separately in memory.
[0062] The sorting cost can be understood as the time consumed during the sorting process, which can be estimated by the query optimizer in the database. The sorting cost when storing at least one row of data to be sorted and its corresponding supplementary data independently in memory can be understood as the time consumed when performing the sorting operations described in S102 to S104. The sorting cost when storing at least one row of data to be sorted together with its corresponding supplementary data in memory can be understood as the time consumed when storing at least one row of data to be sorted together with its corresponding supplementary data in memory and then sorting using the jointly stored data to be sorted and supplementary data (i.e., without performing the method steps shown in S102 to S104). Those skilled in the art will understand that storing data to be sorted together with supplementary data and then sorting it is common knowledge, and this embodiment will not elaborate on it further.
[0063] It should be noted that the two adaptive judgment methods provided above can be used in combination. When used in combination, as long as one of the two conditions is met, it can be guaranteed that the sorting time by executing the method steps shown in S102 to S104 is lower. That is, as long as the size of the data to be sorted is smaller than the size of the CPU cache, or the sorting cost when storing the data separately in memory is less than the sorting cost when storing it together in memory, the method steps shown in S102 to S104 can be executed, thereby quickly outputting the sorting result.
[0064] In some embodiments, predicting the sorting cost takes more time than predicting the size of the data to be sorted. Therefore, the method of predicting the size of the data to be sorted described above can be used first. If the size of the data to be sorted is greater than or equal to the size of the CPU cache, then the sorting cost can be predicted, thereby reducing the time consumed in the sorting method selection process.
[0065] To facilitate understanding, the following will combine... Figure 2 The following examples illustrate the specific implementation methods of the embodiments in this specification.
[0066] Specifically, Figure 2 This is a schematic diagram illustrating the specific process of a data sorting method provided in an exemplary embodiment. Please refer to... Figure 2 After receiving the user's query sorting instruction (select c1,c2,c3 from t order by c1), "c1" can be designated as the sorting key in the data table, and "c2" and "c3" can be designated as additional fields.
[0067] Subsequently, an adaptive judgment can be made regarding the data stored in "c1", "c2", and "c3". If the data stored in the sort key "c1" is less than the CPU's L3 cache, or if the sorting cost of storing the sort key "c1" separately from the additional fields "c2" and "c3" is less than the sorting cost of storing the sort key "c1" together with the additional fields "c2" and "c3", then it can be determined that sorting using the separate storage method is faster and more efficient. Therefore, the sorting task for the sort key "c1" and the additional fields "c2" and "c3" meets the adaptive judgment condition, and sorting can be performed using the separate storage method.
[0068] Therefore, the data in the sort key "c1" and the data in the additional fields "c2" and "c3" can be encoded into sortable data types and materialized and stored in memory. Specifically, the data in the sort key "c1" and the data in the additional fields "c2" and "c3" are stored separately in memory in a row-oriented manner. An additional pointer field is added to the row-oriented sort key data. This additional pointer field stores the memory address of each row of the additional fields in the row-oriented storage, thereby maintaining the correspondence between the data in the sort key "c1" and the data in the additional fields "c2" and "c3".
[0069] The data in sort key "c1", which carries an additional pointer field, is stored in the CPU's L3 cache so that this part of the data is sorted in the L3 cache according to a preset rule. Then, the sorting result is returned to memory, and the additional pointer field in the sorting result is replaced with the additional data corresponding to the memory address it is stored in. This merges the sorted data in sort key "c1" with the data in additional fields "c2" and "c3" to obtain the complete sorting result.
[0070] Figure 3 This is a schematic diagram of the structure of a device provided in an exemplary embodiment. Please refer to... Figure 3 At the hardware level, the device includes a processor 302, an internal bus 304, a network interface 306, memory 308, and non-volatile memory 310, and may also include other hardware required for its functions. One or more embodiments of this specification can be implemented in software, for example, the processor 302 reads the corresponding computer program from the non-volatile memory 310 into memory 308 and then runs it. Of course, in addition to software implementation, one or more embodiments of this specification do not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. That is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.
[0071] Please refer to Figure 4 , Figure 4 A data sorting device 400 is provided, which can be applied to, for example Figure 3 The device shown implements the technical solution of this specification. For example, the data sorting apparatus 400 may include:
[0072] The determination module 401 is used to determine a sort key and an additional field associated with the sort key in a data table. The sort key is used to store at least one row of data to be sorted, and the additional field is used to store additional data corresponding to each row of data to be sorted.
[0073] Storage module 402 is used to independently store the at least one row of data to be sorted and the corresponding additional data into memory.
[0074] The sorting module 403 is used to sort at least one row of data to be sorted stored in memory according to a preset sorting rule.
[0075] The merging module 404 is used to merge the additional data stored in memory with at least one row of sorted data to obtain the sorted result.
[0076] In some embodiments, the sorting module 403 is specifically used to: store at least one row of data to be sorted stored in the memory into the CPU cache; sort the at least one row of data to be sorted stored in the CPU cache according to a preset sorting rule; and return the sorted at least one row of data to be sorted to the memory.
[0077] In some embodiments, the data sorting apparatus 400 further includes a first prediction module (not shown in the figure). The first prediction module is used to predict the data size of the at least one row of data to be sorted. The storage module 402 is specifically used to, when the data size is smaller than the size of the CPU cache, independently store the at least one row of data to be sorted and the corresponding additional data into memory.
[0078] In some embodiments, the data sorting apparatus 400 further includes a second prediction module (not shown in the figures). The second prediction module is used to predict the sorting cost when the at least one row of data to be sorted and the corresponding supplementary data are stored independently in memory, and the sorting cost when the at least one row of data to be sorted is stored together with the corresponding supplementary data in memory. Specifically, the storage module 402 is used to store the at least one row of data to be sorted and the corresponding supplementary data independently in memory when the sorting cost when storing them independently in memory is less than the sorting cost when storing them together in memory.
[0079] In some embodiments, the storage module 402 is specifically used to store the at least one row of data to be sorted and the corresponding additional data in memory in a row-based manner, wherein each row of data to be sorted in the row-based storage is provided with an additional pointer field, which is used to store the memory address of the additional data corresponding to each row of data to be sorted in memory.
[0080] In some embodiments, the merging module 404 is specifically used to replace the additional pointer field in at least one row of sorted data with the corresponding additional data stored in memory to obtain the sorting result.
[0081] In some embodiments, the CPU cache is a Level 3 cache of the CPU.
[0082] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which can take the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email sending and receiving device, game console, tablet computer, wearable device, or any combination of these devices.
[0083] In a typical configuration, a computer includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0084] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.
[0085] Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage, quantum memory, graphene-based storage media or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.
[0086] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0087] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.
[0088] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of one or more embodiments of this specification. The singular forms “a,” “described,” and “the” used in one or more embodiments of this specification and in the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more associated listed items.
[0089] It should be understood that although the terms first, second, third, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first information may also be referred to as second information without departing from the scope of one or more embodiments of this specification, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "in response to a determination," or "when," or "in the event of a determination."
[0090] The above description is merely a preferred embodiment of one or more embodiments of this specification and is not intended to limit the scope of one or more embodiments of this specification. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of one or more embodiments of this specification should be included within the protection scope of one or more embodiments of this specification.
Claims
1. A data sorting method, comprising: In a data table, a sort key and an additional field associated with the sort key are determined. The sort key is used to store at least one row of data to be sorted, and the additional field is used to store additional data corresponding to each row of data to be sorted. Storing the at least one row of data to be sorted and the corresponding supplementary data independently into memory includes: predicting the size of the at least one row of data to be sorted; if the size of the data to be sorted is less than the size of the CPU cache, storing the at least one row of data to be sorted and the corresponding supplementary data independently into memory; if the size of the data to be sorted is greater than or equal to the size of the CPU cache, predicting the sorting cost when storing the at least one row of data to be sorted and the corresponding supplementary data independently into memory, and the sorting cost when storing the at least one row of data to be sorted together with the corresponding supplementary data into memory; if the sorting cost when storing them independently into memory is less than the sorting cost when storing them together into memory, storing the at least one row of data to be sorted and the corresponding supplementary data independently into memory; wherein, each row of data to be sorted includes an supplementary pointer field, which stores the memory address of the supplementary data corresponding to each row of data to be sorted in memory; Sort at least one row of data to be sorted stored in memory according to a preset sorting rule; Based on the additional pointer field, the additional data stored in memory is merged with at least one row of sorted data to obtain the sorting result.
2. The method according to claim 1, wherein sorting at least one row of data to be sorted stored in memory according to a preset sorting rule comprises: Store at least one row of data to be sorted in the memory into the CPU cache; According to the preset sorting rules, sort at least one row of unsorted data stored in the CPU cache; Return at least one row of sorted data to the memory.
3. The method according to claim 1, wherein merging the additional data stored in memory with at least one row of sorted data to obtain a sorting result includes: Replace the additional pointer field in at least one row of unsorted data after sorting with the corresponding additional data stored in memory to obtain the sorting result.
4. The method according to claim 2 or 3, wherein the CPU cache is a level 3 cache of the CPU.
5. A data sorting device, comprising: A determination module is used to determine a sort key and an additional field associated with the sort key in a data table. The sort key is used to store at least one row of data to be sorted, and the additional field is used to store additional data corresponding to each row of data to be sorted. A storage module is configured to independently store the at least one row of data to be sorted and corresponding supplementary data into memory, including: predicting the size of the at least one row of data to be sorted; if the size of the data to be sorted is less than the size of the CPU cache, storing the at least one row of data to be sorted and corresponding supplementary data into memory independently; if the size of the data to be sorted is greater than or equal to the size of the CPU cache, predicting the sorting cost when storing the at least one row of data to be sorted and corresponding supplementary data into memory independently, and the sorting cost when storing the at least one row of data to be sorted together with the corresponding supplementary data into memory; and storing the at least one row of data to be sorted and corresponding supplementary data into memory independently if the sorting cost when storing them independently is less than the sorting cost when storing them together into memory; wherein, each row of data to be sorted includes an supplementary pointer field, which stores the memory address of the supplementary data corresponding to each row of data to be sorted in memory; The sorting module is used to sort at least one row of data to be sorted stored in memory according to a preset sorting rule; The merging module is used to merge the additional data stored in memory with at least one row of sorted data based on the additional pointer field to obtain the sorting result.
6. An electronic device, comprising: processor; Memory used to store processor-executable instructions; The processor executes the executable instructions to implement the method as described in any one of claims 1 to 4.
7. A computer-readable storage medium having stored thereon computer instructions that, when executed by a processor, implement the steps of the method as claimed in any one of claims 1 to 4.