Data partitioning and reorganization methods and related products for column-oriented databases

Abstract

This invention provides a data partitioning reorganization method for column-oriented databases and related products. The data partitioning reorganization method includes: obtaining a new partition instruction; determining the partition to be processed from multiple hash partitions of the column-oriented database based on the new partition instruction; determining a data migration strategy based on the data to be migrated out of the partition to be processed; using the data migration strategy, inserting the data to be migrated out of the partition to the target new partition, and deleting the data to be migrated out of the partition to be processed from the partition to be processed. This method optimizes the process of adding partitions in a column-oriented database, improving overall processing efficiency and reducing storage and computing resource overhead.

Description

Technical Field

[0001] This invention relates to the field of database technology, and in particular to a data partitioning and reorganization method for column-oriented databases, computer-readable storage media, computer program products, and computer devices. Background Technology

[0002] Hash partitioning is a partitioning method that determines data ownership by calculating a hash value for the partition key and then taking the modulo. In most database systems, hash partitioning is used to solve data skew problems, and is particularly suitable for scenarios where data values ​​are discontinuous and their growth is unpredictable. Currently, hash partitioning typically supports dynamically adding hash partitions, i.e., a hash-based partition creation mechanism. The basic logic of this mechanism is to first filter data from the old partitions that meets the criteria for creating the new partition, then insert that data into the new partition, and finally delete that data from the old partitions. After this round of data reorganization between the old and new partitions, the hash partition creation operation is complete.

[0003] In column-store databases, data is stored column-wise, rather than row-wise as in traditional row-store databases. This storage method has significant advantages in querying specific columns and data compression because it can efficiently read and process data sets of specified columns. A batch in a column-store database contains data from multiple rows.

[0004] In the process of adding a new partition in hash partitioning, the related technology requires sorting all rows in the batch data when deleting existing data from the old partition. This operation has a fixed sorting overhead of O(n log n) and consumes additional memory. Especially when the data volume is small, this leads to a significant waste of unnecessary storage and computing resources. Summary of the Invention

[0005] One object of the present invention is to provide a data partitioning and reorganization method for column-store databases, a computer-readable storage medium, a computer program product, and a computer device to optimize the process of adding new partitions in column-store databases, thereby improving overall processing efficiency and reducing storage and computing resource overhead.

[0006] Specifically, according to one aspect of the present invention, the present invention provides a data partitioning reorganization method for a column-store database, comprising: Get the command to add a new partition; Based on the newly added partition instruction, a partition to be processed is determined from multiple hash partitions of the column storage database. The partition to be processed is the hash partition containing data to be migrated out. Based on the data to be migrated from the partition to be processed, a data migration strategy is determined, which is used to migrate data between the partition to be processed and the target new partition. Using the data migration strategy, the data to be migrated out of the partition to be processed is inserted into the target new partition, and the data to be migrated out of the partition to be processed is deleted from the partition to be processed.

[0007] Optionally, determining the data migration strategy based on the data to be migrated from the partition to be processed includes: Obtain a batch of data from the partition to be processed, and use it as the current batch of data; Obtain the amount of data to be migrated out in the current batch of data; Obtain the data volume threshold corresponding to the current batch of data; The data migration strategy for the current batch of data is determined based on the amount of data to be migrated and the data volume threshold.

[0008] Optionally, determining the data migration strategy for the current batch of data based on the amount of data to be migrated and the data volume threshold includes: Determine whether the amount of data to be migrated is less than the data amount threshold; If it is less than, then the data migration strategy is determined to be a single-row migration strategy, which is used to migrate the data to be migrated out of the current batch of data row by row. If it is greater than or equal to, then the data migration strategy is determined to be a batch migration strategy, which is used to perform batch migration of multiple rows of data to be migrated out of the current batch of data.

[0009] Optionally, the single-row migration strategy is used to insert the data to be migrated out of the partition to be processed into the target new partition, and the data to be migrated out of the partition to be processed is deleted from the partition to be processed, including: Traverse the data rows in the current batch of data, and take the traversed data row as the current data row; Obtain the partition hash value of the current data row and determine whether the current data row is mapped to the target new partition; If mapped to the target new partition, then the current data row is inserted into the target new partition; Delete the current data row from the partition to be processed.

[0010] Optionally, deleting the current data row from the partition to be processed includes: Obtain the tuple identifier parameter of the current data row; Using the tuple identifier parameter, the position of the current data row in the partition to be processed is obtained; Delete the current data row from the partition to be processed.

[0011] Optionally, the batch migration strategy is used to insert the data to be migrated out of the partition to be processed into the target new partition, and to delete the data to be migrated out of the partition to be processed from the partition to be processed, including: Calculate the partition hash value of each data row in the current batch of data, and take all data rows that are mapped to the target new partition as the current set of rows to be migrated; Insert the current set of trips to be migrated into the target new partition; Batch delete the current set of trips to be migrated from the partition to be processed.

[0012] Optionally, the step of batch deleting the current set of trips to be migrated from the partition to be processed includes: Sort the current batch of data; Based on the current set of travel destinations to be migrated, update the compressed unit structure information bitmap of the current batch of data to mark each current travel destination to be migrated as deleted in the compressed unit structure information bitmap.

[0013] According to another aspect of the present invention, a computer-readable storage medium is also provided, on which a computer program is stored, wherein the computer program, when executed by a processor, implements the steps of the data partitioning and reorganization method for any of the above-described column-store databases.

[0014] According to another aspect of the present invention, a computer program product is also provided, comprising a computer program that, when executed by a processor, implements the steps of the data partitioning and reorganization method for any of the above-described column-store databases.

[0015] According to another aspect of the present invention, a computer device is also provided, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the data partitioning and reorganization method for any of the above-described column-store databases.

[0016] The data partitioning and reorganization method for column-oriented databases of the present invention obtains the data to be migrated from the partition to be processed and dynamically determines the most suitable data migration strategy based on the characteristics of the data to be migrated. In application scenarios where the data to be migrated is small and scattered, it effectively reduces storage and computing resource overhead and improves data processing speed. In application scenarios where the data to be migrated is large and concentrated, it effectively improves overall processing efficiency, increases processing speed, and enhances resource utilization.

[0017] The above and other objects, advantages and features of the present invention will become more apparent to those skilled in the art from the following detailed description of specific embodiments of the invention in conjunction with the accompanying drawings. Attached Figure Description

[0018] The following sections will describe some specific embodiments of the invention in detail by way of example and not limitation, with reference to the accompanying drawings. The same reference numerals in the drawings denote the same or similar parts or portions. Those skilled in the art should understand that these drawings are not necessarily drawn to scale. In the drawings: Figure 1 This is a flowchart illustrating a data partitioning and reorganization method according to an embodiment of the present invention; Figure 2 This is a flowchart illustrating the process of determining a data migration strategy in a data partitioning and reorganization method according to an embodiment of the present invention. Figure 3 This is a flowchart illustrating the determination of a single-row migration strategy or a batch migration strategy in a data partitioning and reorganization method according to an embodiment of the present invention. Figure 4 This is a schematic diagram of the data migration process using a single-row migration strategy in a data partitioning and reorganization method according to an embodiment of the present invention; Figure 5 This is a flowchart illustrating the data partitioning and reorganization method according to an embodiment of the present invention, which uses a single-row migration strategy to delete data. Figure 6 This is a schematic diagram of the data migration process using a batch migration strategy in a data partitioning and reorganization method according to an embodiment of the present invention; Figure 7 This is a flowchart illustrating the process of deleting data using a batch migration strategy in a data partitioning and reorganization method according to an embodiment of the present invention. Figure 8 This is a flowchart illustrating a data partitioning and reorganization method according to another embodiment of the present invention; Figure 9 This is a schematic diagram of a computer program product according to an embodiment of the present invention; Figure 10 This is a schematic diagram of a computer-readable storage medium according to an embodiment of the present invention; and Figure 11 This is a schematic diagram of a computer device according to an embodiment of the present invention. Detailed Implementation

[0019] The purpose of the data partitioning and reorganization method for column-store databases in this embodiment is to optimize the process of adding new partitions in column-store databases, thereby improving overall processing efficiency and reducing the overhead of storage and computing resources.

[0020] Figure 1This is a flowchart illustrating a data partitioning and reorganization method for a column-store database according to an embodiment of the present invention. The method generally includes: S100, retrieve the command to add a new partition; S200, based on the new partition instruction, determines the partition to be processed from multiple hash partitions of the column storage database. The partition to be processed is the hash partition containing data to be migrated out. S300: Based on the data to be migrated from the partition to be processed, a data migration strategy is determined. The data migration strategy is used to migrate data between the partition to be processed and the target new partition. S400 uses a data migration strategy to insert the data to be migrated from the partition to be processed into the target new partition, and deletes the data to be migrated from the partition to be processed.

[0021] The ALTER TABLE ADDPARTITION command is used to add a hash partition to a columnstore hash partitioned table.

[0022] Column-oriented databases store data using a column-oriented storage schema. A hash partitioned table can include multiple hash partitions (i.e., old partitions), each storing one or more batches of data, with each batch consisting of multiple rows. When a new partition is added to a hash partitioned table, the existing data needs to be redistributed according to the new hash rules. Generally, if the hash partitioned table has a large number of partitions or a small amount of data, the amount of data that needs to be migrated from each old partition will be small; conversely, a large amount of data will need to be migrated.

[0023] In this embodiment, after obtaining the instruction to add a new partition, the initialization work for the batch insertion of the new partition can be performed first, and then the hash partition set can be traversed to select the hash partitions containing data to be migrated out as the partitions to be processed.

[0024] Next, each partition to be processed is processed sequentially. Specifically, data can be obtained by scanning the partitions to be processed, and the data to be migrated out can be filtered according to the new hashing rules.

[0025] Next, a data migration strategy is determined based on the data to be migrated from the partition to be processed.

[0026] For example, in scenarios characterized by "small and scattered" data, such as a large number of partitions to be processed, a small total amount of data in the partitions, a small amount of data to be migrated out of the partitions, and a small ratio of data to be migrated out to the total amount of data in the partitions to be processed, a single-row migration strategy can be used. Specifically, data rows to be migrated out can be filtered row by row, inserted into their corresponding new target partitions, and then the data rows can be deleted from the partitions to be processed. In this way, it is not necessary to sort the data rows in the partitions to be processed, thereby reducing resource consumption and system overhead.

[0027] For example, in scenarios characterized by "large and concentrated" data, such as a small number of partitions to be processed, a large total amount of data in the processed partitions, a large amount of data to be migrated out of the processed partitions, and a large ratio of data to be migrated out to the total amount of data in the processed partitions, inserting and deleting row by row is inefficient. In such cases, a batch migration strategy can be used. Specifically, multiple data rows can be selected in batches from the data to be migrated out, and these multiple data rows can be inserted into the corresponding target new partitions in batches. Then, these data rows can be deleted in batches from the partitions to be processed. Compared to deleting row by row, although batch deletion of multiple data rows triggers sorting of the data in the partitions to be processed, adding additional sorting overhead, the overall deletion efficiency is higher, the speed is faster, and the resource utilization is also higher.

[0028] For example, when the distribution of data to be migrated out in each partition is uneven, or when the distribution of data to be migrated out in different batches of data in the partition is uneven, a data migration strategy can be dynamically and flexibly determined. Specifically, for batches of data with a small amount of data to be migrated out, a single-row migration strategy can be used. For batches of data with a large amount of data to be migrated out, a batch migration strategy can be used.

[0029] This embodiment of the column-oriented database data partitioning reorganization method obtains the data to be migrated from the partition to be processed and dynamically determines the most suitable data migration strategy based on the characteristics of the data to be migrated. In application scenarios where the data to be migrated is small and scattered, it effectively reduces storage and computing resource overhead and improves data processing speed. In application scenarios where the data to be migrated is large and concentrated, it effectively improves overall processing efficiency, increases processing speed, and enhances resource utilization.

[0030] In some embodiments of the data partitioning and reorganization method of the present invention, such as Figure 2 As shown, based on the data to be migrated from the partition to be processed, the data migration strategy is determined, including: S311, retrieve a batch of data from the partition to be processed and use it as the current batch of data; S313, Get the amount of data to be migrated out in the current batch of data; S315, obtain the data volume threshold corresponding to the current batch of data; S317, determine the data migration strategy for the current batch of data based on the amount of data to be migrated and the data volume threshold.

[0031] In this embodiment, instead of formulating a data migration strategy for each partition to be processed, a data migration strategy is formulated for each batch of data in the partition to be processed, thereby further refining and optimizing the steps and improving the targeting of the data migration strategy.

[0032] A batch of data typically includes a preset number of rows (usually hundreds, thousands, or tens of thousands). In this embodiment, the amount of data to be migrated out can be the number of rows to be migrated out in the current batch of data. The data volume threshold can be set according to the preset number of rows in the batch of data; for example, the data volume threshold can be set to 1 / 10 or 1 / 5 of the preset number of rows in the batch of data. The data volume threshold can also be set in conjunction with the hardware performance of the database system; when the database system has strong data sorting capabilities, a lower data volume threshold can be used.

[0033] In some embodiments of the data partitioning and reorganization method of the present invention, such as Figure 3 As shown, based on the amount of data to be migrated and the data volume threshold, the data migration strategy for the current batch of data is determined, including: S321, Determine whether the amount of data to be migrated is less than the data amount threshold; S323, if it is less than, then the data migration strategy is determined to be a single-row migration strategy. The single-row migration strategy is used to migrate the data to be migrated out of the current batch of data row by row. S325, if greater than or equal to, then the data migration strategy is determined to be a batch migration strategy. The batch migration strategy is used to perform batch migration of multiple rows of data to be migrated out of the current batch of data.

[0034] This embodiment introduces a threshold control strategy, which can automatically select the optimal operation method based on the size of the amount of data to be migrated in the current batch: a single-row migration strategy is used for small batches of data to ensure efficiency; while a more efficient batch migration strategy is used for large batches of data. This ensures flexibility, improves overall resource utilization, and reduces resource consumption.

[0035] In some embodiments of the data partitioning and reorganization method of the present invention, such as Figure 4 As shown, a single-row migration strategy is used to insert the data to be migrated from the partition to be processed into the target new partition, and then delete the data to be migrated from the partition to be processed. This includes: S411, Traverse the data rows in the current batch of data and use the traversed data row as the current data row; S413, obtain the partition hash value of the current data row, and determine whether the current data row is mapped to the target new partition; S415, If mapped to a new target partition, insert the current data row into the new target partition; S417, Delete the current data row from the partition to be processed.

[0036] For example, a column-store database includes four hash partitions, p0-p3. When adding a new partition p4, the current batch of data can be retrieved from p0. This current batch of data may contain, for example, five data rows. During the single-row migration strategy, the partition hash value of row 1 is first calculated to determine if migration is unnecessary. Then, the partition hash value of row 2 is calculated to determine if migration is unnecessary. Next, the partition hash value of row 3 is calculated, and if migration to the new target partition p4 is determined, row 3 is inserted into p4 and then deleted from p0. The partition hash value of row 4 is then calculated, and if migration to the new partition p4 is determined, row 4 is inserted into p4 and then deleted from p0. Finally, the partition hash value of row 5 is calculated, and if migration is not necessary, migration is not required. This embodiment avoids sorting within the current batch of data in p0 by deleting migrated data rows one by one, reducing resource consumption and system overhead.

[0037] In some embodiments of the data partitioning and reorganization method of the present invention, such as Figure 5 As shown, deleting the current data row from the partition to be processed includes: S421, retrieve the tuple identifier parameter for the current data row; S423, using the tuple identifier parameter, obtains the position of the current data row in the partition to be processed; S425, Delete the current data row from the partition to be processed.

[0038] In this embodiment, after the data of the current data row is inserted into the target new partition, the position of the current data row in the current batch of data in p0 can be accurately located according to its tuple identifier (tid), and then it can be deleted to reduce storage usage.

[0039] In some embodiments of the data partitioning and reorganization method of the present invention, such as Figure 6 As shown, a batch migration strategy is used to insert the data to be migrated from the partition to be processed into the target new partition, and to delete the data to be migrated from the partition to be processed, including: S441, calculate the partition hash value of each data row in the current batch of data, and take all data rows that are mapped to the target new partition as the current set of rows to be migrated; S443, insert the current set of trips to be moved into the target new partition; S445, Batch delete the current set of trips to be migrated from the partition to be processed.

[0040] In this embodiment, all data rows of the current batch of data are traversed, and the partition hash value of each data row is calculated to filter out the current rows to be migrated, thus obtaining the current set of rows to be migrated. Then, the current set of rows to be migrated is inserted into the target new partition in batches, and the current set of rows to be migrated is deleted from the partition to be processed in batches. Although deleting multiple data rows in batches will trigger the sorting of the data in the partition to be processed, adding additional sorting overhead, the overall deletion efficiency is high, the speed is fast, and the resource utilization is also high.

[0041] In some embodiments of the data partitioning and reorganization method of the present invention, such as Figure 7 As shown, the set of currently pending migration lines is deleted in batches from the partition to be processed, including; S451, Sort the data in the current batch; S453, based on the current set of trips to be migrated, update the compressed unit structure information bitmap of the current batch of data, so as to mark each current trip to be migrated as deleted in the compressed unit structure information bitmap.

[0042] In this embodiment, the deletion operation in column storage is a batch deletion. Specifically, the current batch of data to be deleted is first obtained, then the current batch of data is sorted, and then the structure information bitmap (CUDesc) of the compression unit (CU) of the current batch of data is updated in batches. Each current row to be migrated is marked as deleted in the compression unit structure information bitmap. In this way, multiple rows of current row data to be migrated can be deleted efficiently, avoiding a large number of I / O operations and reducing computational overhead.

[0043] In some embodiments of the data partitioning and reorganization method of the present invention, such as Figure 8 As shown, data partitioning and reorganization methods include: S511, retrieve the command to add a new partition; S512, obtain the partition to be processed; S513, Scan the partition to be processed, use the new hash partitioning rules to filter out the data rows to be migrated that are mapped to the target new partition, and insert them into the target new partition; S514, determine whether the data volume of the data row to be migrated is less than the preset data volume threshold; if it is less, execute S515; if it is greater than or equal to, execute S516. S515: Based on the tid of each data row to be migrated, determine the location in the partition to be processed and delete the data; S516 stores each data row to be migrated and deletes them in batches from the partition to be processed.

[0044] The flowchart provided in this embodiment is not intended to indicate that the operations of the method will be performed in any particular order, or that all operations of the method are included in every case. Furthermore, the method may include additional operations. Within the scope of the technical concept provided by the method in this embodiment, additional variations can be made to the above method.

[0045] It should be understood that in some embodiments, the components may be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods may be implemented using software or firmware stored in memory and executed by a suitable instruction execution system.

[0046] This invention also provides a computer program product 10, a computer-readable storage medium 20, and a computer device 30. Figure 9 This is a schematic diagram of a computer program product 10 according to an embodiment of the present invention. Figure 10 This is a schematic diagram of a computer-readable storage medium 20 according to an embodiment of the present invention. Figure 11 This is a schematic diagram of a computer device 30 according to an embodiment of the present invention. The computer program product 10 includes a computer program 11, which, when executed by the processor 32, implements the steps of any of the data partitioning and reorganization methods described above. A computer-readable storage medium 20 stores the computer program 11 thereon, which, when executed by the processor 32, implements the steps of the data partitioning and reorganization method of any of the above embodiments. The computer device 30 may include a memory 31, a processor 32, and the computer program 11 stored in the memory 31 and running on the processor 32.

[0047] The computer program 11 used to perform the operations of this invention may be assembly instructions, Instruction Set Architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, integrated circuit configuration data, or source code or object code written in any combination of one or more programming languages ​​and procedural programming languages. The computer program 11 may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network, including a Local Area Network (LAN) or Wide Area Network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, to perform aspects of this invention, electronic circuits, including, for example, programmable logic circuits, Field-Programmable Gate Arrays (FPGAs), or Programmable Logic Arrays (PLAs), may execute computer-readable program instructions to personalize the electronic circuits by utilizing state information from computer-readable program instructions.

[0048] For the purposes of this embodiment, computer program product 10 is a related product that includes computer program 11.

[0049] For the purposes of this embodiment, the computer-readable storage medium 20 is a tangible device capable of holding and storing a computer program 11. It can be any device capable of containing, storing, communicating, propagating, or transmitting the computer program 11 for use by or in conjunction with an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable storage medium 20 include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable optical disc read-only memory (CD-ROM), digital versatile disc (DVD), memory stick, floppy disk, mechanical encoding device, and any suitable combination thereof.

[0050] Computer device 30 can be, for example, a server, desktop computer, laptop computer, tablet computer, or smartphone. In some examples, computer device 30 can be a cloud computing node. Computer device 30 can be described in the general context of computer system executable instructions (such as program modules) executed by a computer system. Typically, program modules can include routines, programs, object programs, components, logic, data structures, etc., that perform specific tasks or implement specific abstract data types. Computer device 30 can be implemented in a distributed cloud computing environment where tasks are performed by remote processing devices linked through a communication network. In a distributed cloud computing environment, program modules can reside on local or remote computing system storage media, including storage devices.

[0051] Computer device 30 may include a processor 32 adapted to execute stored instructions and a memory 31 that provides temporary storage space for the operation of said instructions during operation. The processor 32 may be a single-core processor, a multi-core processor, a computing cluster, or any other configuration. The memory 31 may include random access memory (RAM), read-only memory, flash memory, or any other suitable storage system.

[0052] Computer device 30 may also include a network adapter / interface and an input / output (I / O) interface. The I / O interface allows external devices that can be connected to the computer device to input and output data. The network adapter / interface provides communication between the computer device and a network, typically represented as a communication network.

[0053] Therefore, those skilled in the art should recognize that although numerous exemplary embodiments of the present invention have been shown and described in detail herein, many other variations or modifications conforming to the principles of the present invention can be directly determined or derived from the disclosure of the present invention without departing from the spirit and scope of the invention. Thus, the scope of the present invention should be understood and construed as covering all such other variations or modifications.

Claims

1. A method for data partitioning and reorganizing a column-oriented database, characterized in that, include: Get the command to add a new partition; Based on the newly added partition instruction, a partition to be processed is determined from multiple hash partitions of the column storage database. The partition to be processed is the hash partition containing data to be migrated out. Based on the data to be migrated from the partition to be processed, a data migration strategy is determined, which is used to migrate data between the partition to be processed and the target new partition. Using the data migration strategy, the data to be migrated out of the partition to be processed is inserted into the target new partition, and the data to be migrated out of the partition to be processed is deleted from the partition to be processed.

2. The data partitioning and reorganization method according to claim 1, characterized in that, The step of determining a data migration strategy based on the data to be migrated from the partition to be processed includes: Obtain a batch of data from the partition to be processed, and use it as the current batch of data; Obtain the amount of data to be migrated out in the current batch of data; Obtain the data volume threshold corresponding to the current batch of data; The data migration strategy for the current batch of data is determined based on the amount of data to be migrated and the data volume threshold.

3. The data partitioning and reorganization method according to claim 2, characterized in that, The step of determining the data migration strategy for the current batch of data based on the amount of data to be migrated and the data volume threshold includes: Determine whether the amount of data to be migrated is less than the data amount threshold; If it is less than, then the data migration strategy is determined to be a single-row migration strategy, which is used to migrate the data to be migrated out of the current batch of data row by row. If it is greater than or equal to, then the data migration strategy is determined to be a batch migration strategy, which is used to perform batch migration of multiple rows of data to be migrated out of the current batch of data.

4. The data partitioning and reorganization method according to claim 3, characterized in that, The single-row migration strategy is used to insert the data to be migrated out of the partition to be processed into the target new partition, and to delete the data to be migrated out of the partition to be processed from the partition to be processed, including: Traverse the data rows in the current batch of data, and take the traversed data row as the current data row; Obtain the partition hash value of the current data row and determine whether the current data row is mapped to the target new partition; If mapped to the target new partition, then the current data row is inserted into the target new partition; Delete the current data row from the partition to be processed.

5. The data partitioning and reorganization method according to claim 4, characterized in that, Deleting the current data row from the partition to be processed includes: Obtain the tuple identifier parameter of the current data row; Using the tuple identifier parameter, the position of the current data row in the partition to be processed is obtained; Delete the current data row from the partition to be processed.

6. The data partitioning and reorganization method according to claim 3, characterized in that, The batch migration strategy is used to insert the data to be migrated out of the partition to be processed into the target new partition, and to delete the data to be migrated out of the partition to be processed from the partition to be processed, including: Calculate the partition hash value of each data row in the current batch of data, and take all data rows that are mapped to the target new partition as the current set of rows to be migrated; Insert the current set of trips to be migrated into the target new partition; Batch delete the current set of trips to be migrated from the partition to be processed.

7. The data partitioning and reorganization method according to claim 6, characterized in that, The step of batch deleting the current set of trips to be migrated from the partition to be processed includes: Sort the current batch of data; Based on the current set of travel destinations to be migrated, update the compressed unit structure information bitmap of the current batch of data to mark each current travel destination to be migrated as deleted in the compressed unit structure information bitmap.

8. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the steps of the data partitioning and reorganization method for a column-store database as described in any one of claims 1 to 7.

9. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the steps of the data partitioning and reorganization method for a column-store database as described in any one of claims 1 to 7.

10. A computer device, characterized in that, The system includes a memory, a processor, and a computer program stored on the memory, wherein the processor executes the computer program to implement the steps of the data partitioning and reorganization method for a column-store database as described in any one of claims 1 to 7.

Images