Data query method and apparatus used in partition splitting process, and electronic device

Abstract

Provided in the present description are a data query method and apparatus used in a partition splitting process, and an electronic device, wherein a data table in a database is partitioned into at least one partition. The method comprises: receiving a data query instruction for a target partition; detecting whether the target partition is a source partition undergoing a partition splitting process; and if the target partition is a source partition undergoing a partition splitting process, determining a destination partition, among at least one destination partition split from the source partition, which corresponds to the data query instruction, and on the basis of data in the destination partition and data in the source partition that belongs to the destination partition, executing the data query instruction, wherein after receiving a splitting instruction for the target partition, the target partition serves as the source partition to perform partition splitting.

Description

Data query method and device applied to partition splitting process, and electronic device TECHNICAL FIELD

[0001] One or more embodiments of the present specification relate to the technical field of database, and in particular to a data query method and device applied to a partition splitting process, and an electronic device. BACKGROUND

[0002] Whether it is traditional Internet technology or emerging artificial intelligence technology, it relies on the support of data, especially the current artificial intelligence technology such as machine learning relying on big data, which makes the database more and more important. In recent years, the amount of data in the database is increasing, which poses a great challenge to the management of the database.

[0003] In the database, especially in the distributed database, the data table can be divided into multiple partitions to achieve data balancing, parallel processing, and other effects. As the amount of data in the partition expands, partition splitting is often needed to prevent the amount of data in the partition from being too large. However, in related technologies, when the partition in the database is split, data query services cannot be provided, that is, data query services cannot be provided externally until the partition splitting is completed SUMMARY

[0004] Therefore, one or more embodiments of the present specification provide a data query method and device applied to a partition splitting process, and an electronic device.

[0005] To achieve the above object, one or more embodiments of the present specification provide the technical solutions as follows:

[0006] According to a first aspect of one or more embodiments of the present specification, a data query method applied to a partition splitting process is provided; wherein a data table in a database is divided into at least one partition; the method comprises:

[0007] receiving a data query instruction for a target partition;

[0008] detecting whether the target partition is a source partition in a partition splitting process;

[0009] If the target partition is a source partition in a partition splitting process, determining at least one destination partition corresponding to the data query instruction from the at least one destination partition split by the source partition, and executing the data query instruction based on the data in the destination partition and the data in the source partition belonging to the destination partition; wherein the target partition, when receiving a splitting instruction for the target partition, performs partition splitting as a source partition.

[0010] In one or more possible embodiments of the present specification, the method further comprises:

[0011] receiving a split instruction for the target partition;

[0012] taking the target partition as a source partition, and creating at least one destination partition as at least one destination partition split from the source partition;

[0013] dividing data in the source partition to the at least one destination partition;

[0014] The detection of whether the target partition is a source partition in a partition splitting process includes:

[0015] detecting whether the target partition is in a process of dividing data in the source partition to the at least one destination partition as the source partition.

[0016] In one or more possible embodiments of the present specification, the execution of the data query instruction based on data in the destination partition and data in the source partition belonging to the destination partition includes:

[0017] determining data in the source partition belonging to the destination partition according to a data boundary of the destination partition, and executing the data query instruction based on data in the destination partition and data in the source partition belonging to the destination partition.

[0018] In one or more possible embodiments of the present specification, the determination of data in the source partition belonging to the destination partition according to the data boundary of the destination partition, and the execution of the data query instruction based on data in the destination partition and data in the source partition belonging to the destination partition include:

[0019] in a case where a data table to which the source partition belongs is a globally indexed table, clipping a data range of the source partition according to the data boundary of the destination partition to obtain a data query range corresponding to the source partition;

[0020] executing the data query instruction based on data in the destination partition and data in the source partition within the data query range.

[0021] In one or more possible embodiments of the present specification, the determination of data in the source partition belonging to the destination partition according to the data boundary of the destination partition, and the execution of the data query instruction based on data in the destination partition and data in the source partition belonging to the destination partition include:

[0022] In a case where the data table to which the source partition belongs is a main table, the data range of the source partition is clipped according to the data boundary of the destination partition, to obtain a data query range corresponding to the source partition;

[0023] The data query instruction is executed based on the data in the destination partition and the data in the source partition within the data query range.

[0024] In one or more possible embodiments of the present specification, the determination of the data in the source partition that belongs to the destination partition according to the data boundary of the destination partition, and the execution of the data query instruction based on the data in the destination partition and the data in the source partition that belongs to the destination partition, include:

[0025] In a case where the data table to which the source partition belongs is a local index table, a data filter is generated according to the data boundary of the destination partition;

[0026] The data in the source partition is filtered based on the data filter to determine the data in the source partition that belongs to the destination partition, and the data query instruction is executed based on the data in the destination partition and the data in the source partition that belongs to the destination partition.

[0027] In one or more possible embodiments of the present specification, the filtering of the data in the source partition based on the data filter to determine the data in the source partition that belongs to the destination partition, and the execution of the data query instruction based on the data in the destination partition and the data in the source partition that belongs to the destination partition, include:

[0028] The data filter is sent to a storage layer, and the data in the source partition is filtered based on the data filter by the storage layer to determine the data in the source partition that belongs to the destination partition, and the data query instruction is executed based on the data in the destination partition and the data in the source partition that belongs to the destination partition.

[0029] In one or more possible embodiments of the present specification, the filtering of the data in the source partition based on the data filter to determine the data in the source partition that belongs to the destination partition, includes:

[0030] For each data row in the source partition, if the data row meets the filtering condition of the data filter, it is determined that the data row does not belong to the destination partition, and if the data row does not meet the filtering condition of the data filter, it is determined that the data row belongs to the destination partition.

[0031] In one or more possible embodiments of this specification, executing the data query instruction based on data in the destination partition and data in the source partition belonging to the destination partition includes:

[0032] Based on the data in the memory table and ordered string table of the destination partition, and the data belonging to the destination partition in the memory table and ordered string table of the source partition, the data query instruction is executed.

[0033] In one or more possible embodiments of this specification, the method further includes:

[0034] If all data within the source partition has been partitioned, the source partition will be deleted.

[0035] According to a second aspect of one or more embodiments of this specification, a data query apparatus for use in a partitioning process is provided; wherein a data table in a database is divided into at least one partition; the apparatus includes:

[0036] The receiving module is used to receive data query commands for the target partition;

[0037] The detection module is used to detect whether the target partition is a source partition that is in the process of partition splitting;

[0038] The query module is configured to, if the target partition is a source partition in the process of partition splitting, determine the target partition corresponding to the data query instruction among at least one target partition split from the source partition, and execute the data query instruction based on the data in the target partition and the data in the source partition belonging to the target partition; wherein, when the target partition receives the split instruction for the target partition, it is used as the source partition for partition splitting.

[0039] In one or more possible embodiments of this specification, the device further includes a splitting module for:

[0040] Receive a split command for the target partition;

[0041] Use the target partition as the source partition, and create at least one destination partition as at least one destination partition split from the source partition;

[0042] Divide the data in the source partition into at least one destination partition;

[0043] The detection module is specifically used for:

[0044] Detect whether the target partition is used as the source partition and is in the process of dividing the data in the source partition to the at least one destination partition.

[0045] In one or more possible embodiments of the present specification, the query module is specifically configured to:

[0046] According to the data boundary of the destination partition, determine the data in the source partition that belongs to the destination partition, and execute the data query instruction based on the data in the destination partition and the data in the source partition that belongs to the destination partition.

[0047] In one or more possible embodiments of the present specification, the query module is specifically configured to:

[0048] In the case where the data table to which the source partition belongs is a global index table, according to the data boundary of the destination partition, the data range of the source partition is cropped to obtain a data query range corresponding to the source partition;

[0049] The data query instruction is executed based on the data in the destination partition and the data in the source partition within the data query range.

[0050] In one or more possible embodiments of the present specification, the query module is specifically configured to:

[0051] In the case where the data table to which the source partition belongs is a master table, according to the data boundary of the destination partition, the data range of the source partition is cropped to obtain a data query range corresponding to the source partition;

[0052] The data query instruction is executed based on the data in the destination partition and the data in the source partition within the data query range.

[0053] In one or more possible embodiments of the present specification, the query module is specifically configured to:

[0054] In the case where the data table to which the source partition belongs is a local index table, according to the data boundary of the destination partition, a data filter is generated;

[0055] The data in the source partition is filtered based on the data filter to determine the data in the source partition that belongs to the destination partition, and the data query instruction is executed based on the data in the destination partition and the data in the source partition that belongs to the destination partition.

[0056] In one or more possible embodiments of the present specification, the query module is specifically configured to:

[0057] The data filter is sent to a storage layer, and the storage layer filters data in the source partition based on the data filter to determine data in the source partition that belongs to the destination partition, and executes the data query instruction based on data in the destination partition and the determined data in the source partition that belongs to the destination partition.

[0058] In one or more embodiments of the present disclosure, the query module is specifically configured to:

[0059] For each data row in the source partition, if the data row meets the filtering condition of the data filter, it is determined that the data row does not belong to the destination partition, and if the data row does not meet the filtering condition of the data filter, it is determined that the data row belongs to the destination partition.

[0060] In one or more embodiments of the present disclosure, the query module is specifically configured to:

[0061] The data query instruction is executed based on data in the memory table and the ordered string table of the destination partition and data in the memory table and the ordered string table of the source partition that belongs to the destination partition.

[0062] In one or more embodiments of the present disclosure, the apparatus further includes a deletion module configured to:

[0063] If all data in the source partition is divided, the source partition is deleted.

[0064] According to a third aspect of one or more embodiments of the present disclosure, a computer program product is provided, including computer programs / instructions, which, when executed by a processor, implement the steps of the method of the first aspect.

[0065] According to a fourth aspect of one or more embodiments of the present disclosure, an electronic device is provided, including:

[0066] a processor;

[0067] a memory for storing processor-executable instructions;

[0068] The processor implements the method of the first aspect by running the executable instructions.

[0069] According to a fifth aspect of one or more embodiments of the present disclosure, a computer-readable storage medium is provided, which stores computer instructions, which, when executed by a processor, implement the steps of the method of the first aspect.

[0070] The technical solutions provided by the embodiments of the present disclosure can include the following beneficial effects:

[0071] The data query method applied in the partition splitting process provided by the embodiment of the present specification is in the process of partition splitting by receiving a splitting instruction for a source partition, creating at least one destination partition, and dividing the data in the source partition to the at least one destination partition, and in the process, if a data query instruction for the source partition is received, the data query instruction can be executed on the basis of the data in the destination partition to which the data query instruction is directed and the data in the source partition corresponding to the destination partition to which the data query instruction is directed, thereby providing data query service externally in the process of partition splitting, avoiding the influence on the user's experience due to service stop, that is, improving the user's experience. BRIEF DESCRIPTION OF DRAWINGS

[0072] FIG. 1 is a structural diagram of a distributed database provided by an example embodiment.

[0073] FIG. 2 is a flowchart of a data query method applied in a partition splitting process provided by an example embodiment.

[0074] FIG. 3 is a transaction processing logic diagram in a partition splitting process provided by an example embodiment.

[0075] FIG. 4 is a processing logic diagram of a data query instruction provided by an example embodiment.

[0076] FIG. 5 is a structural diagram of a device provided by an example embodiment.

[0077] FIG. 6 is a block diagram of a data query device applied in a partition splitting process provided by an example embodiment. DETAILED DESCRIPTION

[0078] The example embodiments will be described in detail herein, with examples shown in the drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following example embodiments do not represent all implementations consistent with one or more embodiments of the present specification. Instead, they are merely examples of apparatuses and methods consistent with some aspects of one or more embodiments of the present specification as detailed in the appended claims.

[0079] It should be noted that the steps of the corresponding method in other embodiments are not necessarily performed in the order shown and described in the present specification. In some other embodiments, the steps included in the method can be more or less than described in the present specification. In addition, a single step described in the present specification can be divided into multiple steps for description in other embodiments, and multiple steps described in the present specification can be combined into a single step for description in other embodiments.

[0080] In the related art, in the process of partition splitting, the source partition stops service, and the target partition is invisible to the outside, so that the service cannot be provided to the outside; until the target partition is visible to the outside after the completion of the partition splitting, the data query service can be provided to the outside.

[0081] Based on the technical problems mentioned in the background, at least one embodiment of the present specification provides a data query method applied in the process of partition splitting, which can provide data query service to the outside in the process of partition splitting, not only ensuring the accuracy of the query result of the data query service, but also not affecting the normal progress of the partition splitting, thereby avoiding affecting the user experience due to the stop of the data query service in the process of partition splitting.

[0082] The method can be applied to a distributed database based on an LSM-Tree storage engine as shown in FIG. 1, or other types of databases.

[0083] It should be noted that for the database, the data table in the database can be divided into at least one partition.

[0084] For example, the data in the above-mentioned database can be organized into an LSM-Tree, which usually includes a MemTable (Memory Table) and an SSTable (Sorted String Table), wherein the MemTable is usually stored in the memory and the SSTable is stored in the disk. Each partition in the database can have its memory table (or a region in the memory table) and sorted string table.

[0085] The data query method applied in the process of partition splitting provided by the present specification can be implemented when a data query instruction for a source partition in the process of partition splitting is received, based on the source partition and at least one target partition split from the source partition, to execute the data query instruction.

[0086] Firstly, the process of partition splitting of a certain partition (which can be referred to as a target partition) in the above-mentioned database as a source partition is described.

[0087] When a classification instruction for the above-mentioned target partition is received, the target partition can start the process of partition splitting. Specifically, the target partition can be used as a source partition, and at least one target partition can be created as at least one target partition split from the source partition; in this case, the data in the source partition can be divided into the at least one target partition.

[0088] Exemplarily, a plurality of destination partitions are constructed according to the number of destination partitions indicated in the split instruction. For example, if the split instruction indicates that the source partition is to be split into two destination partitions, then two destination partitions are constructed.

[0089] Exemplarily, after receiving the split instruction, the source partition can be stopped from writing and the data in the source partition that is not completed is written to the instruction execution completion; and the memory table in the source partition is frozen and written to the disk to form an ordered string table, and then the data in the source partition is divided into the at least one destination partition, or the memory table in the source partition is frozen, and then the data in the source partition is divided into the at least one destination partition. It should be understood that if the source partition is stopped from writing, and then a data write instruction is received, the data write instruction can be forwarded to the corresponding destination partition for execution.

[0090] Exemplarily, according to the mapping relationship between the data and the destination partitions indicated in the split instruction, each data in the source partition is divided into the corresponding destination partition, for example, each data in the source partition is copied (replicated) to the corresponding destination partition. For example, if the source partition with a key range of [1, 100) is split into a destination partition 1 with a key range of [1, 50) and a destination partition 2 with a key range of [51, 100), then the data in the source partition in the key range [1, 50) is divided into the destination partition 1, and the data in the source partition in the key range [51, 100) is divided into the destination partition 2.

[0091] It should be understood that if all the data in the source partition is completed, the source partition can be deleted.

[0092] Please refer to FIG. 2, which exemplarily shows the flow of the data query method applied in the partition splitting process, including steps S201 to S203.

[0093] In step S201, a data query instruction for a target partition is received.

[0094] For the above target partition, a data query instruction for the target partition can be received.

[0095] In step S202, it is detected whether the target partition is a source partition in a partition splitting process.

[0096] In the case that the data query instruction for the above target partition is received, it can be detected whether the target partition is a source partition in a partition splitting process.

[0097] For example, when receiving a split instruction for the target partition, the target partition can be taken as a source partition, and at least one destination partition can be created as at least one destination partition split from the source partition, and then data in the source partition can be divided into the at least one destination partition. Correspondingly, it can be detected whether the target partition is in the process of dividing data in the source partition into the at least one destination partition as the source partition, and if yes, it indicates that the target partition is the source partition in the partition splitting process, otherwise, it can be considered that the target partition is not the source partition in the partition splitting process.

[0098] In step S203, if the target partition is the source partition in the partition splitting process, a destination partition corresponding to the data query instruction in at least one destination partition split from the source partition is determined, and the data query instruction is executed based on data in the destination partition and data in the source partition belonging to the destination partition. When receiving a split instruction for the target partition, the target partition is taken as a source partition for partition splitting.

[0099] If the target partition is not the source partition in the partition splitting process, the data query instruction for the target partition can be executed normally. That is, the data query instruction can be executed based on data in the target partition.

[0100] If the target partition is the source partition in the partition splitting process, the data query instruction can be executed by referring to data in a destination partition to which the data query instruction is directed and data in the source partition corresponding to the destination partition to which the data query instruction is directed. That is, a destination partition corresponding to the data query instruction of the source partition in at least one destination partition split from the source partition (i.e., the target partition) can be determined first, and the data query instruction can be executed based on data in the destination partition and data in the source partition belonging to the destination partition.

[0101] It should be noted that, at this time, the data in the destination partition corresponding to the data query instruction of the source partition refers to data that has been divided from the source partition to the destination partition, and the data in the source partition belonging to the destination partition refers to data that should be divided from the source partition to the destination partition but has not been really divided from the source partition to the destination partition.

[0102] After the source partition starts the partition splitting, it stops external services, including data writing services and data query services. When the partition splitting is not completed, the destination partition does not have all the corresponding data (i.e., data that should be stored in the destination partition after the completion of the partition splitting), and therefore the data in the source partition needs to be referred to.

[0103] It should be understood that the stop-write of the source partition when receiving the data query instruction in this step has ended. For example, the source partition ends the stop-write after freezing the memory table, and this step can execute the data query instruction based on the data in the memory table and the ordered string table of the destination partition to which the data query instruction is directed, and the data in the memory table and the ordered string table of the source partition corresponding to the destination partition to which the data query instruction is directed; that is, the data query instruction can be executed based on the data in the memory table and the ordered string table of the destination partition to which the data query instruction is directed, and the data in the memory table and the ordered string table of the source partition belonging to the destination partition to which the data query instruction is directed. For another example, the source partition ends the stop-write after freezing the memory table and writing to the disk to form the ordered string table, and this step can execute the data query instruction based on the data in the memory table and the ordered string table of the destination partition to which the data query instruction is directed, and the data in the ordered string table of the source partition corresponding to the destination partition to which the data query instruction is directed; that is, the data query instruction can be executed based on the data in the memory table and the ordered string table of the destination partition to which the data query instruction is directed, and the data in the ordered string table of the source partition belonging to the destination partition to which the data query instruction is directed.

[0104] Please refer to FIG. 3, which shows an exemplary transaction processing logic diagram in the partition splitting process of the present specification. The old transaction (old trans) of the client table scan is sent to the source partition (Old Partition (Parent)) of the database node for processing before the partition splitting; and the source partition (Old Partition (Parent)) starts to split into two destination partitions (New Partition (Daughter)), and the new transaction (new trans) of the client table scan is sent to the corresponding destination partition (New Partition (Daughter)) in the database node for partition stream cutting during the partition splitting process. The destination partition (New Partition (Daughter)) does not have corresponding complete data, so it needs to obtain the missing data (get new partition data) from the source partition (Old Partition (Parent)) and refer to the data in the destination partition (New Partition (Daughter)) and the obtained data to process the above new transaction (new trans).

[0105] The method for querying data in a partition splitting process provided by the embodiments of the present specification can be used to receive a splitting instruction for a source partition, create at least one destination partition, and divide data in the source partition into the at least one destination partition to be in a process of partition splitting. If a data query instruction for the source partition is received in the process, the data query instruction can be executed on the basis of data in a destination partition to which the data query instruction is directed and data in the source partition corresponding to the destination partition to which the data query instruction is directed, that is, the data query instruction is executed on the basis of data in the destination partition to which the data query instruction is directed and data in the source partition belonging to the destination partition to which the data query instruction is directed, so that data query services are provided externally in the process of partition splitting, and the use experience of users is not affected due to service stop, that is, the use experience of users is improved.

[0106] In some embodiments of the present disclosure, step S202 can be performed in the following manner: the data query instruction is executed on the basis of data in a destination partition to which the data query instruction is directed and a result of filtering data in the source partition on the basis of a data boundary of the destination partition to which the data query instruction is directed. Specifically, data in the source partition belonging to the destination partition to which the data query instruction is directed can be determined according to the data boundary of the destination partition to which the data query instruction is directed, and the data query instruction can be executed on the basis of data in the destination partition and data in the source partition belonging to the destination partition.

[0107] Specifically, the above execution manner provided by the present embodiment can be completed in the following optional examples.

[0108] Optional Example 1

[0109] When data in the source partition belonging to the destination partition to which the data query instruction is directed is determined according to the data boundary of the destination partition to which the data query instruction is directed, and the data query instruction is executed on the basis of data in the destination partition and data in the source partition belonging to the destination partition, specifically, in the case where a data table to which the source partition belongs is a global index table, the data range of the source partition can be clipped according to the data boundary of the destination partition to obtain a data query range corresponding to the source partition, and then the data query instruction can be executed on the basis of data in the destination partition and data in the source partition located in the data query range.

[0110] Similarly, when the data in the source partition belonging to the destination partition according to the data boundary of the destination partition to which the data query instruction is directed is determined, and the data query instruction is executed based on the data in the destination partition and the data in the source partition belonging to the destination partition, specifically, when the data table to which the source partition belongs is a master table, the data range of the source partition can be clipped according to the data boundary of the destination partition to obtain a data query range corresponding to the source partition, and then the data query instruction can be executed based on the data in the destination partition and the data in the source partition within the data query range.

[0111] In actual application, first, when the data table to which the source partition belongs is a global index table, the data range of the source partition is clipped based on the data boundary of the destination partition to which the data query instruction is directed to obtain a scanning range. The scanning range can represent a data query range corresponding to the source data. The primary key of the global index table is ordered, and the data boundary of the destination partition in partition splitting is also represented by the primary key. Therefore, the data range of the source partition can be partitioned and clipped based on the data boundary (i.e., the left and right boundaries) of the destination partition to obtain the scanning range.

[0112] Next, the data in the source partition and the data in the destination partition to which the data query instruction is directed are obtained, and the data in the source partition and the data in the destination partition to which the data query instruction is directed are scanned based on the scanning range to obtain the data in the scanning range. For example, if the source partition ends after the memory table is frozen, the data in the memory table and the ordered string table of the destination partition to which the data query instruction is directed, and the data in the memory table and the ordered string table of the source partition corresponding to the destination partition to which the data query instruction is directed are scanned based on the scanning range; or if the source partition ends after the memory table is frozen and written to the disk to form an ordered string table, the data in the memory table and the ordered string table of the destination partition to which the data query instruction is directed, and the data in the ordered string table of the source partition corresponding to the destination partition to which the data query instruction is directed are scanned based on the scanning range.

[0113] Finally, the data query instruction is executed based on the data in the scanning range.

[0114] Optional Example 2

[0115] In a case where the data table to which the source partition belongs is a local index table, a data filter can be generated based on the data boundary of the destination partition, and the data in the source partition can be filtered based on the data filter to determine the data in the source partition belonging to the destination partition, and the data query instruction can be executed based on the data in the destination partition and the determined data in the source partition belonging to the destination partition.

[0116] In actual applications, first, in a case where the data table to which the source partition belongs is a local index table, a data filter can be generated based on the data boundary of the destination partition to which the data query instruction is directed.

[0117] In the local index table, the data rows are sorted based on a non-primary key, that is, the primary key is unordered, and thus the partition pruning cannot be directly performed in the manner of the optional example 1. The data filter can be an expression representing a filtering condition.

[0118] Next, the data in the source partition and the data in the destination partition to which the data query instruction is directed are obtained, and the data in the source partition and the data in the destination partition to which the data query instruction is directed are filtered based on the data filter to obtain data that is not filtered out by the data filter. The data that is not filtered out by the data filter includes not only the data in the destination partition to which the data query instruction is directed, but also the data in the source partition belonging to the destination partition to which the data query instruction is directed.

[0119] For example, the data filter can be sent to a storage layer, and the data in the source partition can be filtered based on the data filter by the storage layer to determine the data in the source partition belonging to the destination partition to which the data query instruction is directed, and the data query instruction can be executed based on the data in the destination partition and the determined data in the source partition belonging to the destination partition.

[0120] For example, for each data row in the source partition, if the data row meets the filtering condition of the data filter, it is determined that the data row does not belong to the destination partition to which the data query instruction is directed, and if the data row does not meet the filtering condition of the data filter, it is determined that the data row belongs to the destination partition to which the data query instruction is directed.

[0121] Specifically, the data filter can be sent to the storage layer to obtain data in the source partition returned by the storage layer and data in the destination partition targeted by the data query instruction and not filtered out by the data filter, wherein the storage layer obtains data in the source partition and data in the destination partition targeted by the data query instruction and filters data in the source partition and data in the destination partition targeted by the data query instruction based on the data filter. By pushing down the data filter to the storage layer, the storage layer can complete data filtering and return only data corresponding to the destination partition to the SQL execution layer, thereby reducing data transmission and occupation of data transmission channels.

[0122] Whether it is the storage layer or the SQL execution layer, when obtaining data in the source partition and data in the destination partition targeted by the data query instruction, if the source partition ends writing after the memory table is frozen, data in the memory table and the ordered string table of the destination partition targeted by the data query instruction and data in the memory table and the ordered string table of the source partition are obtained; if the source partition ends writing after the memory table is frozen and written to the disk to form an ordered string table, data in the memory table and the ordered string table of the destination partition targeted by the data query instruction and data in the ordered string table of the source partition are obtained.

[0123] Whether it is the storage layer or the SQL execution layer, when filtering data in the source partition and data in the destination partition targeted by the data query instruction based on the data filter, for each data row in the source partition and the destination partition targeted by the data query instruction, if the data row meets the filtering condition of the data filter, the data row is filtered out, and if the data row does not meet the filtering condition of the data filter, the data row is retained.

[0124] Finally, the data query instruction is executed based on the data not filtered out by the data filter.

[0125] Please refer to FIG. 4, which exemplarily shows the logical relationship between the processing manners provided by the various examples.

[0126] When scanning a partition in a table scan, it is determined whether the partition is splitting. If the partition is not splitting, the partition is scanned and output. If the partition is splitting, it is determined whether the table to which the partition belongs is an is local index. If the table is not an is local index, the scan range is cut, and the source partition and the destination partition are scanned based on the cut scan range to output. If the table is an is local index, the source partition and the destination partition are obtained, and the source partition and the destination partition are scanned and filtered to output.

[0127] FIG. 5 is a schematic structural diagram of an apparatus according to an example embodiment. Referring to FIG. 5, at the hardware level, the apparatus includes a processor 502, an internal bus 504, a network interface 506, a memory 508, and a non-volatile memory 510, and can further include other hardware required by tasks. One or more embodiments of the present specification can be implemented in a software manner, for example, by the processor 502 reading a corresponding computer program from the non-volatile memory 510 into the memory 508 and then running. Of course, in addition to the software implementation, one or more embodiments of the present specification do not exclude other implementation manners, such as a logic device or a combination of software and hardware, and the like, that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or a logic device.

[0128] Referring to FIG. 6, the data query device applied in the partition splitting process can be applied in the apparatus shown in FIG. 5 to implement the technical solutions of the present specification. In the data query device, the data table in the database can be divided into at least one partition, and the data query device applied in the partition splitting process can include:

[0129] The receiving module 601 is configured to receive a data query instruction for a target partition;

[0130] The detection module 602 is configured to detect whether the target partition is a source partition in a partition splitting process;

[0131] The query module 603 is configured to: if the target partition is a source partition in a partition splitting process, determine a target partition corresponding to the data query instruction from at least one target partition split from the source partition, and execute the data query instruction based on data in the target partition and data in the source partition belonging to the target partition; and wherein the target partition is split as a source partition when a splitting instruction for the target partition is received.

[0132] In one or more possible embodiments of the present specification, the apparatus further comprises a splitting module configured to:

[0133] receive a splitting instruction for the target partition;

[0134] split the target partition as a source partition, and create at least one target partition as at least one target partition split from the source partition;

[0135] divide data in the source partition into the at least one target partition;

[0136] The detection module is specifically configured to:

[0137] detect whether the target partition is in a process of dividing data in the source partition into the at least one target partition as the source partition.

[0138] In one or more possible embodiments of the present specification, the query module is specifically configured to:

[0139] determine data in the source partition belonging to the target partition according to a data boundary of the target partition, and execute the data query instruction based on data in the target partition and data in the source partition belonging to the target partition.

[0140] In one or more possible embodiments of the present specification, the query module is specifically configured to:

[0141] in a case where a data table to which the source partition belongs is a globally indexed table, clip a data range of the source partition according to a data boundary of the target partition to obtain a data query range corresponding to the source partition;

[0142] execute the data query instruction based on data in the target partition and data in the source partition within the data query range.

[0143] In one or more possible embodiments of the present specification, the query module is specifically configured to:

[0144] In a case where the data table to which the source partition belongs is a primary table, the data range of the source partition is clipped according to the data boundary of the destination partition, to obtain a data query range corresponding to the source partition;

[0145] The data query instruction is executed based on the data in the destination partition and the data in the source partition within the data query range.

[0146] In one or more possible embodiments of the present specification, the query module is specifically configured to:

[0147] In a case where the data table to which the source partition belongs is a local index table, a data filter is generated according to the data boundary of the destination partition;

[0148] The data in the source partition is filtered based on the data filter to determine the data in the source partition belonging to the destination partition, and the data query instruction is executed based on the data in the destination partition and the data in the source partition determined to belong to the destination partition.

[0149] In one or more possible embodiments of the present specification, the query module is specifically configured to:

[0150] The data filter is sent to a storage layer, and the data in the source partition is filtered based on the data filter to determine the data in the source partition belonging to the destination partition, and the data query instruction is executed based on the data in the destination partition and the data in the source partition determined to belong to the destination partition.

[0151] In one or more possible embodiments of the present specification, the query module is specifically configured to:

[0152] For each data row in the source partition, if the data row meets the filtering condition of the data filter, it is determined that the data row does not belong to the destination partition, and if the data row does not meet the filtering condition of the data filter, it is determined that the data row belongs to the destination partition.

[0153] In one or more possible embodiments of the present specification, the query module is specifically configured to:

[0154] The data query instruction is executed based on the data in the memory table and the ordered string table of the destination partition and the data in the memory table and the ordered string table of the source partition belonging to the destination partition.

[0155] In one or more possible embodiments of the present specification, the apparatus further includes a deletion module configured to:

[0156] If all data in the source partition is completed partitioning, the source partition is deleted.

[0157] The systems, apparatuses, modules or units disclosed in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, and the specific form of the computer can be a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0158] In a typical configuration, a computer includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0159] The memory can include non-persistent memory in computer readable media, random access memory (RAM), and / or non-volatile memory such as read-only memory (ROM) or Flash memory. The memory is an example of computer readable media.

[0160] Computer readable media includes permanent and non-permanent, removable and non-removable media implemented by any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, disk storage, quantum memory, graphene-based storage medium or other magnetic storage device, or any other non-transmission medium that can be used to store information accessible by a computing device. According to the definition herein, computer readable media does not include transitory media such as modulated data signals and carriers.

[0161] It should also be noted that the terms "comprising", "including", or any other variant thereof are intended to cover non-exclusive inclusion, so that processes, methods, articles or devices including a series of elements not only include those elements, but also include other elements not explicitly listed or inherent to such processes, methods, articles or devices. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or device including the element.

[0162] The above-described embodiments of the application have several advantages. For example, the above-described embodiments of the application provide a method and system for providing a user with a personalized experience. Other embodiments of the application have other benefits and advantages.

[0163] The terminology used in this description is for the purpose of describing particular embodiments only and is not intended to limit one or more embodiments of the present description. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0164] The user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present description are information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation portal for user to choose authorization or refusal.

[0165] It should be understood that, although one or more embodiments of the present description can employ the terms first, second, third, etc. to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, the first information can also be referred to as the second information, and similarly, the second information can also be referred to as the first information, without departing from the scope of one or more embodiments of the present description. Depending on the context, the word "if" as used herein can be interpreted as "when" or "upon" or "in response to determining".

[0166] The above only describes the preferred embodiments of one or more embodiments of the present description, and does not limit one or more embodiments of the present description. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of one or more embodiments of the present description should be included in the protection scope of one or more embodiments of the present description.

Claims

1. A method for data query applied in a partition splitting process; wherein, The data table in the database is divided into at least one partition; the method comprises: receiving a data query instruction for a target partition; detecting whether the target partition is a source partition in a partition splitting process; if the target partition is a source partition in a partition splitting process, determining a destination partition corresponding to the data query instruction from at least one destination partition split from the source partition, and executing the data query instruction based on data in the destination partition and data in the source partition belonging to the destination partition; wherein the target partition, when receiving a splitting instruction for the target partition, performs partition splitting as a source partition.

2. The method of claim 1, further comprising: receiving a splitting instruction for the target partition; taking the target partition as a source partition and creating at least one destination partition as at least one destination partition split from the source partition; dividing data in the source partition into the at least one destination partition; the detection of whether the target partition is a source partition in a partition splitting process comprises: detecting whether the target partition is in the process of dividing data in the source partition into the at least one destination partition as the source partition.

3. The data query method applied in a partition splitting process according to claim 1, wherein the execution of the data query instruction based on data in the destination partition and data in the source partition belonging to the destination partition comprises: determining data in the source partition belonging to the destination partition according to a data boundary of the destination partition, and executing the data query instruction based on data in the destination partition and data in the source partition belonging to the destination partition.

4. The data query method applied in a partition splitting process according to claim 3, wherein the determination of data in the source partition belonging to the destination partition according to a data boundary of the destination partition, and the execution of the data query instruction based on data in the destination partition and data in the source partition belonging to the destination partition comprises: in a case where a data table to which the source partition belongs is a globally indexed table, clipping a data range of the source partition according to the data boundary of the destination partition to obtain a data query range corresponding to the source partition; executing the data query instruction based on data in the destination partition and data in the source partition within the data query range.

5. The data query method applied in a partition splitting process according to claim 3, wherein the determination of data in the source partition belonging to the destination partition according to a data boundary of the destination partition, and the execution of the data query instruction based on data in the destination partition and data in the source partition belonging to the destination partition comprises: in a case where a data table to which the source partition belongs is a primary table, clipping a data range of the source partition according to the data boundary of the destination partition to obtain a data query range corresponding to the source partition; execute the data query instruction based on the data in the target partition and the data in the source partition that belongs to the target partition. 6.The data query method applied to a partition splitting process according to claim 3, wherein the determining the data in the source partition that belongs to the target partition according to the data boundary of the target partition and the executing the data query instruction based on the data in the target partition and the data in the source partition that belongs to the target partition comprise: in a case where a data table to which the source partition belongs is a locally indexed table, generating a data filter according to the data boundary of the target partition; and filtering the data in the source partition based on the data filter to determine the data in the source partition that belongs to the target partition, and executing the data query instruction based on the data in the target partition and the data in the source partition that belongs to the target partition. 7.The data query method applied to a partition splitting process according to claim 6, wherein the filtering the data in the source partition based on the data filter to determine the data in the source partition that belongs to the target partition and the executing the data query instruction based on the data in the target partition and the data in the source partition that belongs to the target partition comprise: sending the data filter to a storage layer, and filtering the data in the source partition based on the data filter to determine the data in the source partition that belongs to the target partition by the storage layer, and executing the data query instruction based on the data in the target partition and the data in the source partition that belongs to the target partition. 8.The data query method applied to a partition splitting process according to claim 6 or 7, wherein the filtering the data in the source partition based on the data filter to determine the data in the source partition that belongs to the target partition comprises: for each data row in the source partition, determining that the data row does not belong to the target partition if the data row meets a filtering condition of the data filter, and determining that the data row belongs to the target partition if the data row does not meet the filtering condition of the data filter. 9.The data query method applied to a partition splitting process according to claim 1, wherein the executing the data query instruction based on the data in the target partition and the data in the source partition that belongs to the target partition comprises: executing the data query instruction based on the data in the in-memory table and the ordered string table of the target partition and the data in the in-memory table and the ordered string table of the source partition that belongs to the target partition. 10.The data query method applied to a partition splitting process according to claim 2, further comprising: deleting the source partition if all data in the source partition is completed splitting. 11.A data query apparatus applied to a partition splitting process, the apparatus comprising: a receiving module configured to receive a data query instruction for a target partition; and a processing module configured to determine data in a source partition that belongs to the target partition according to a data boundary of the target partition, and execute the data query instruction based on the data in the target partition and the data in the source partition that belongs to the target partition. ​ ​ ​ ​ ​ ​ ​ detecting whether the target partition is a source partition in a partition splitting process; querying, if the target partition is a source partition in a partition splitting process, a destination partition corresponding to the data query instruction from at least one destination partition split from the source partition, and executing the data query instruction based on data in the destination partition and data in the source partition belonging to the destination partition; wherein the target partition, when receiving a splitting instruction for the target partition, performs partition splitting as a source partition. 12.A computer program product comprising computer programs / instructions which, when executed by a processor, implement the steps of the method of any one of claims 1 to 10. 13.An electronic device comprising: a processor; a memory for storing processor-executable instructions; wherein the processor implements the method of any one of claims 1 to 10 by running the executable instructions. 14.A computer-readable storage medium having stored thereon computer instructions which, when executed by a processor, implement the steps of the method of any one of claims 1 to 10.

Images