Data splitting method, device, equipment, medium and program product

Abstract

The present application provides a data splitting method, device, equipment, medium and program product, which determines at least one data source node in the database cluster to be split according to preset requirements; then sends the data to be split in the database cluster to be split to the data source node; and splits the data to be split in the data source node according to the preset splitting method to determine the split data; finally, the split data is sent to at least one target node of at least one target database cluster through the data source node. It solves the technical problems in the prior art that data splitting involves many aspects of changes, the splitting code has low reusability, and changing the table splitting logic also requires modifying the query code on the business side. It achieves the technical effect of efficient data splitting without changing the table splitting logic, high splitting code reusability, and no need to change the query code on the business side.

Description

Technical Field

[0001] The present application relates to the field of computer databases, and in particular to a data splitting method, device, equipment, medium and program product. Background Art

[0002] With the continuous development of Internet technology, various Internet businesses are also increasing rapidly. However, when most businesses go online, it is impossible to predict the data scale of the business in the later stage. Therefore, most businesses use a single database instance cluster to carry the business during development. However, as the business develops, the business volume grows rapidly. The amount of data in some data tables is too large, resulting in an overall decline in the query and update performance of the system, which in turn affects the overall business throughput of the system.

[0003] When dealing with this problem, the existing technology generally splits a data table with a complex header structure into multiple data tables with simple header structures, or uses database-based replication technology to split data tables with large amounts of data into new databases to improve the overall operating efficiency of the entire system.

[0004] However, the above splitting method, whether it is splitting a table or splitting a database, has poor versatility in the splitting process and is complicated. When the business continues to grow, the same problem will arise when splitting again, and it is also necessary to modify the SQL (Structured Query Language) code when the business party accesses the database. That is, the prior art has technical problems such as data splitting involving multiple changes, low reusability of the splitting code, and the need to modify the database access code on the business side at the same time. Summary of the invention

[0005] The present application provides a data splitting method, device, equipment, medium and program product to solve the technical problems in the prior art that data splitting involves multiple changes, the splitting code has low reusability, and the database access code needs to be modified on the business side at the same time.

[0006] In a first aspect, the present application provides a data splitting method, comprising:

[0007] In the database cluster to be split, at least one data source node is determined according to preset requirements, and the data source node is an intermediate medium for data replication and transfer between the database cluster to be split and the target database cluster;

[0008] Sending the data to be split in the database cluster to be split to the data source node;

[0009] Splitting the data to be split in the data source node according to a preset splitting method to determine split data, wherein the preset splitting method retains the logical structure of the original data table, that is, the preset splitting method is used to split the data without changing the header structure of the original data table;

[0010] The split data is sent to a target node of a target database cluster through the data source node.

[0011] In one possible design, the database cluster to be split includes at least one master-slave relationship node, and the master-slave relationship node includes: a master node and at least one slave node attached to the master node, and the data source node includes at least one of the slave nodes in the master-slave relationship node.

[0012] In a possible design, splitting the to-be-split data in the data source node according to a preset splitting method to determine split data includes:

[0013] The split data is determined according to preset splitting parameters corresponding to preset key fields in the data to be split.

[0014] Optionally, the preset splitting parameters include: a splitting interval range and at least one splitting value within the splitting interval range, and determining the split data according to the preset splitting parameters corresponding to the preset key fields in the data to be split includes:

[0015] Keep the header structure of the data table fixed, determine the split data according to the split interval range and the split value corresponding to the preset key field in the data table, the split data has the same structure as the data table, and the data to be split includes at least one of the data tables.

[0016] In a possible design, sending the split data to a target node of a target database cluster through the data source node includes:

[0017] Creating a write queue for each of the target nodes;

[0018] Using a read coroutine to read the split data from the data source node, and inserting the split data into the write queue;

[0019] The split data is copied from the write queue to the corresponding target node in sequence using a write coroutine.

[0020] In a possible design, after sending the data to be split in the database cluster to be split to the data source node, the method further includes:

[0021] The data source node is removed from the database cluster to be split so that the total amount of data in the data source node remains unchanged.

[0022] In a possible design, after sending the split data to at least one target node of at least one target database cluster through the data source node, the method further includes:

[0023] Reconnecting the data source node to the database cluster to be split;

[0024] Sending the updated data to be split to the data source node;

[0025] Splitting the data to be split again according to a preset splitting method to determine new split data;

[0026] The new split data is sent to the target node through the data source node in a filtering and copying manner, and the filtering and copying manner is used to filter out the split data that already exists in the target node.

[0027] Optionally, the preset requirement includes: the data source node is a read-only node.

[0028] Further optionally, the read-only node is a read-only slave node in the master-slave relationship node.

[0029] In a possible design, sending the data to be split in the database cluster to be split to the data source node includes:

[0030] According to the attribute characteristics of the data to be split, first data to be split that meets the preset conditions is sent to the data source node in a preset sending manner, and the data to be split includes the first data to be split.

[0031] Optionally, the sending, according to the attribute characteristics of the data to be split, the first data to be split that meets the preset conditions to the data source node in a preset sending manner includes:

[0032] Sending the second data to be split and the third data to be split to the data source node in a preset order, wherein the attribute feature of the second data to be split is greater than or equal to a preset feature threshold, and the attribute feature of the third data to be split is less than the preset feature threshold;

[0033] The first data to be split includes the second data to be split and the third data to be split.

[0034] Optionally, the second data to be split and the third data to be split are sent to the data source node in a preset order, including:

[0035] First, sending one of the second data to be split and the third data to be split to the data source node;

[0036] After sending the split data corresponding to the second data to be split or the third data to be split to the target node through the data source node, the other one is sent to the data source node.

[0037] In a possible design, sending the data to be split in the database cluster to be split to the data source node includes:

[0038] Arranging the data to be split according to the requirements of preset attribute characteristics to determine a queue of data to be split;

[0039] According to the arrangement order of the queue of data to be split, the data to be split is sent to the data source node in batches; wherein,

[0040] After the previous batch of data is split in the data source node and the split data is sent to the target node through the data source node, the next batch of data is sent to the data source node.

[0041] In a possible design, when the number of the data source nodes is at least two, sending the data to be split in the database cluster to be split to the data source nodes includes:

[0042] The data to be split are distributed to different data source nodes according to attribute characteristics of the data to be split.

[0043] Optionally, the attribute feature includes: a first attribute feature and a second attribute feature, and distributing the data to be split to different data source nodes according to the attribute feature of the data to be split includes:

[0044] Sending fourth data to be split to the first data source node, wherein a first attribute feature of the fourth data to be split meets a first feature requirement;

[0045] Sending fifth data to be split to the second data source node, wherein the second attribute feature of the fifth data to be split meets the second feature requirement;

[0046] The data to be split include: the fourth data to be split and the fifth data to be split, and the data source nodes include: the first data source node and the second data source node.

[0047] In a possible design, after sending the split data to at least one target node of at least one target database cluster through the data source node, the method further includes:

[0048] Verify whether the data in the target node has been correctly copied;

[0049] If yes, switching part or all of the services of the database cluster to be split to the target database cluster;

[0050] If not, correct the corresponding problem data.

[0051] In a second aspect, the present application provides a data splitting device, comprising:

[0052] A source node selection module, used to determine at least one data source node in the database cluster to be split according to preset requirements, wherein the data source node is an intermediate medium for data replication and transfer between the database cluster to be split and the target database cluster;

[0053] A data preparation module for splitting, used for sending the data to be split in the database cluster to be split to the data source node, wherein the preset splitting method retains the logical structure of the original data table;

[0054] A splitting module, used for splitting the data to be split in the data source node according to a preset splitting method to determine split data;

[0055] The splitting module is further used to send the split data to at least one target node of at least one target database cluster through the data source node.

[0056] In one possible design, the database cluster to be split includes at least one master-slave relationship node, and the master-slave relationship node includes: a master node and at least one slave node attached to the master node, and the data source node includes at least one of the slave nodes in the master-slave relationship node.

[0057] In a possible design, the splitting module is specifically used to determine the split data according to preset splitting parameters corresponding to preset key fields in the data to be split.

[0058] Optionally, the preset splitting parameters include: a splitting interval range and at least one splitting value within the splitting interval range, and the splitting module is specifically configured to:

[0059] Keeping the structure of the data table unchanged, the split data is determined according to the split interval range and the split value corresponding to the preset key field in the data table, the split data has the same structure as the data table, and the data to be split includes at least one of the data tables.

[0060] In a possible design, the splitting module is further specifically used for:

[0061] Creating a write queue for each of the target nodes;

[0062] Using a read coroutine to read the split data from the data source node, and inserting the split data into the write queue;

[0063] The split data is copied from the write queue to the corresponding target node in sequence using a write coroutine.

[0064] In a possible design, the splitting module is also used to remove the data source node from the database cluster to be split, so that the total amount of data in the data source node remains unchanged.

[0065] In a possible design, the splitting module is further used for:

[0066] Reconnecting the data source node to the database cluster to be split;

[0067] Sending the updated data to be split to the data source node;

[0068] Splitting the data to be split again according to a preset splitting method to determine new split data;

[0069] The new split data is sent to the target node through the data source node in a filtering and copying manner, and the filtering and copying manner is used to filter out the split data that already exists in the target node.

[0070] Optionally, the preset requirement includes: the data source node is a read-only node.

[0071] Further optionally, the read-only node is a read-only slave node in the master-slave relationship node.

[0072] In a possible design, the data to be split preparation module is used to send the first data to be split that meets preset conditions to the data source node in a preset sending manner according to attribute characteristics of the data to be split, and the data to be split includes the first data to be split.

[0073] Optionally, the module for preparing data to be split is specifically used for:

[0074] Sending the second data to be split and the third data to be split to the data source node in a preset order, wherein the attribute feature of the second data to be split is greater than or equal to a preset feature threshold, and the attribute feature of the third data to be split is less than the preset feature threshold;

[0075] The first data to be split includes the second data to be split and the third data to be split.

[0076] Optionally, the module for preparing data to be split is specifically used for:

[0077] First, sending one of the second data to be split and the third data to be split to the data source node;

[0078] After sending the split data corresponding to the second data to be split or the third data to be split to the target node through the data source node, the other one is sent to the data source node.

[0079] In a possible design, the to-be-split data preparation module is used to arrange the to-be-split data according to the requirements of preset attribute characteristics to determine a to-be-split data queue;

[0080] The to-be-split data preparation module is further configured to send the to-be-split data to the data source node in batches according to the arrangement order of the to-be-split data queue; wherein,

[0081] After the previous batch of data is split in the data source node and the split data is sent to the target node through the data source node, the next batch of data is sent to the data source node.

[0082] In a possible design, when the number of the data source nodes is at least two, the module for preparing the data to be split is used to distribute the data to be split to different data source nodes according to attribute characteristics of the data to be split.

[0083] Optionally, the attribute feature includes: a first attribute feature and a second attribute feature, and the to-be-split data preparation module is used to send the fourth to-be-split data to the first data source node, and the first attribute feature of the fourth to-be-split data meets the first feature requirement;

[0084] The to-be-split data preparation module is further configured to send fifth to-be-split data to the second data source node, wherein the second attribute feature of the fifth to-be-split data meets the second feature requirement;

[0085] The data to be split include: the fourth data to be split and the fifth data to be split, and the data source nodes include: the first data source node and the second data source node.

[0086] In a possible design, the data splitting device further includes:

[0087] A verification module, used to verify whether the data in the target node has been correctly copied;

[0088] If so, then

[0089] A switching module, used to switch part or all of the services of the database cluster to be split to the target database cluster;

[0090] If not, then

[0091] The splitting module is also used to correct corresponding problem data.

[0092] In a third aspect, the present application provides an electronic device, including:

[0093] A memory for storing program instructions;

[0094] The processor is used to call and execute the program instructions in the memory to execute any possible method for determining item storage information provided in the first aspect.

[0095] In a fourth aspect, the present application provides a storage medium, wherein the readable storage medium stores a computer program, and the computer program is used to execute any possible data splitting method provided in the first aspect.

[0096] In a fifth aspect, the present application also provides a computer program product, including a computer program, which, when executed by a processor, implements any possible data splitting method provided in the first aspect.

[0097] The present application provides a data splitting method, device, equipment, medium and program product, which determines at least one data source node in the database cluster to be split according to preset requirements; then sends the data to be split in the database cluster to be split to the data source node; and splits the data to be split in the data source node according to the preset splitting method to determine the split data; finally, the split data is sent to at least one target node of at least one target database cluster through the data source node. It solves the technical problems in the prior art that data splitting involves many aspects of changes, the splitting code has low reusability, and changing the table splitting logic also requires modifying the query code on the business side. It achieves the technical effect of efficient data splitting without changing the table splitting logic, high splitting code reusability, and no need to change the query code on the business side. BRIEF DESCRIPTION OF THE DRAWINGS

[0098] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0099] Figure 1a-1c A schematic diagram of an application scenario for keeping the logical structure of a data table unchanged provided in an embodiment of the present application;

[0100] Figure 2 A flowchart of a data splitting method provided in an embodiment of the present application;

[0101] Figure 3 A schematic diagram of the structure of a database cluster to be split provided in an embodiment of the present application;

[0102] Figure 4 A schematic diagram of the structure of another database cluster to be split provided in an embodiment of the present application;

[0103] Figure 5 A schematic diagram of constructing a subsequence from a logical sequence provided in an embodiment of the present application;

[0104] Figure 6 For the embodiment of this application Figure 2 A schematic diagram of the implementation process of S202 in the data splitting method shown;

[0105] Figure 7 A flowchart of another data splitting method provided for the implementation of this application;

[0106] Figure 8 A schematic diagram of the structure of a data splitting device provided in an embodiment of the present application;

[0107] Fig. 9 A schematic diagram of the structure of an electronic device provided in this application.

[0108] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0109] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work, including but not limited to the combination of multiple embodiments, belong to the scope of protection of this application.

[0110] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein, for example. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0111] When most businesses go online, it is impossible to predict the data scale in the later stage of the business, so a single database instance cluster is used to carry the business. As the business develops, the business volume grows rapidly, and the data volume of some tables is too large, resulting in an overall decline in query and update performance, which in turn affects the overall throughput of the system.

[0112] At this time, splitting is needed, and there are many ways to split the database. In the prior art, when splitting database data, one is to split a data table with a complex header structure into multiple new data tables with simple header structures, such as splitting the t table into t1~t16 or more tables to provide services; the other is to split the data table with large data volume separately into a new database, and the system version of the new database is updated, and the database capacity is larger, that is, the new database has a stronger reading and writing ability for the data table with large data volume.

[0113] However, a common problem in the prior art is that the splitting code is very poor in universality. Each split requires developers to rewrite the splitting code or manually modify the existing splitting code. In addition, whether it is splitting a table or splitting a database, it is inevitable to modify the SQL read and write code for accessing the database on the business side. That is, the prior art has the technical problem that data splitting involves multiple changes, the splitting code has low reusability, and the database access code also needs to be modified on the business side.

[0114] In order to solve the above technical problems, the invention concept of this application is:

[0115] Data tables are the basic elements of a database. The essence of business-side access to a database is to perform a series of operations on data tables, such as adding, deleting, reading, and modifying. In other words, if you do not want to change the access code on the business side or the business logic, you cannot change the data table.

[0116] However, the data volume in the data table is too large, which affects the access efficiency of the business side and requires us to modify the data table. This seems to be a contradictory and unsolvable problem.

[0117] After in-depth analysis, the inventor of the present application found that the reason why the technical personnel in the field believe that the above problem has no solution is that the technical personnel in the field have the inertial thinking of "changing the data table". The technical personnel in the field have the inherent thinking that the data table is a visible table, and the vertical or horizontal splitting of the table will be disassembled into multiple new data tables, and the original table will no longer exist.

[0118] However, the inventor of this application has broken this cognitive inertia and thought from the essence of the data table. The essence of the data table does not actually depend on the physical table to exist, that is, the data table in the database is not equivalent to the table we print on paper. A data table is actually a data set with a preset logical structure. It is the inherent connection between the logical structure and the data itself that is the constraint condition for judging whether a data element belongs to the set of data tables.

[0119] In other words, no matter where the data is stored or in what form, as long as the logical structure of the data table itself remains unchanged, the data table will remain unchanged. The fundamental purpose of the so-called split is not to tear up the data table or reconstruct the data table, but to enable the business end to quickly locate the data when accessing it. As long as the logic of the split table remains unchanged, the business end naturally does not need to rewrite the access code.

[0120] In this way, we transform the problem into how to manage the data stored in each new database with the logical structure of the data table in the original database, that is, the table splitting logic, so that no matter how many times the data table is split, the logical data table will never change. And this management method should not be too complicated to avoid consuming too much computing power.

[0121] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0122] First, let us introduce an implementation method that keeps the table partitioning logic, that is, the logical structure of the data table unchanged.

[0123] Figure 1a-1c A schematic diagram of an application scenario for keeping the logical structure of a data table unchanged provided in an embodiment of the present application. Figure 1aAs shown, the data table 100 to be split contains a large number of data records, and the header structure can be extracted separately to form a first logical sequence 101, and the following data records are matched with the second logical sequence 102. In this way, the logical structure of the data table 100 is formed by the first logical sequence 101 and the second logical sequence 102, or the data table 100 can be represented by the first logical sequence 101 and the second logical sequence 102.

[0124] It should be noted that each element value in the first logical sequence 101 and the second logical sequence 102 may be a coded value obtained by a preset coding method, such as coding by a hash algorithm. Then the access code of the service side can correspond to the first logical sequence 101 and the second logical sequence 102 during compilation.

[0125] Then the split data table 100 can be converted into the splitting of the first logical sequence 101 and / or the second logical sequence 102, that is, the first logical sequence 101 and / or the second logical sequence 102 can contain multiple subsequences.

[0126] Figure 1b and Figure 1c That is, the first logical sequence 101 remains unchanged, and the second logical sequence 102 is horizontally split to obtain two parts of the data table 100 stored in the new database. Note that there are two parts here instead of two tables, because the present application splits the logical sequence instead of the table. This is the difference between the present application and the prior art. For example, the prior art will give each table a separate file when distinguishing tables. The splitting of tables in the prior art will increase the number of files, which is why it is necessary to modify the business side access code. The logical sequence of the present application is not a data table, but it can restore the table, or the first logical sequence 101 and the second logical sequence 102 establish a virtual table. No matter how the actual data storage is split, the virtual table is a whole.

[0127] Next, the data splitting method for keeping the logical structure of the data table unchanged is introduced in detail.

[0128] Figure 2 A flowchart of a data splitting method provided in an embodiment of the present application is shown in FIG. Figure 2 As shown, the specific steps of the data splitting method include:

[0129] S201. In a database cluster to be split, determine at least one data source node according to preset requirements.

[0130] In this step, the data source node is an intermediate medium for data replication and transfer between the database cluster to be split and the target database cluster.

[0131] In this embodiment, the database cluster to be split includes at least one master-slave relationship node, and the master-slave relationship node includes: a master node and at least one slave node attached to the master node, and the data source node includes at least one of the slave nodes in the master-slave relationship node.

[0132] In a possible implementation, the preset requirement includes: the data source node is a read-only node. That is, the type of the data source node is a read-only node. This can avoid the situation where the data is accidentally deleted or modified due to uncertain factors during data splitting.

[0133] Figure 3 This is a schematic diagram of the structure of a database cluster to be split provided in an embodiment of the present application. Figure 3 As shown, there is only one master-slave relationship node 300 in the database cluster to be split, wherein the data stored in the master node 301 and the slave node 302 are consistent, but the slave node 302 is only responsible for providing complex query, data analysis and other services to the outside, and cannot perform write operations, and the operation of writing or changing data must access the master node 301 to complete. And the master node 301 performs data synchronization with the slave node 302 regularly or in real time to ensure that the data of the two nodes are consistent. In principle, any one of the master node 301 and the slave node 302 can be arbitrarily selected as the data source node of the data splitting, but in order to avoid the wrong operation of deleting or modifying the data in the original database cluster during the splitting and copying process, the slave node is generally selected as the real-time data source node of the data splitting method of this embodiment.

[0134] If the database cluster to be split contains multiple nodes and the relationship between the nodes is complex, such as Figure 4 shown.

[0135] Figure 4 This is a schematic diagram of another structure of a database cluster to be split provided in an embodiment of the present application. Figure 4As shown, the database cluster to be split includes: an independent node 401, a first master-slave relationship node 402, a second master-slave relationship node 403, and a third master-slave relationship node 404. Among them, the first master-slave relationship node 402 includes: a first master node 4021 and a first slave node 4022; the second master-slave relationship node 403 includes: a second master node 4031, a second slave node 4032, and a third slave node 4033; the third master-slave relationship node 404 includes: a third master node 4041 and a fourth slave node 4042. In principle, any node can be arbitrarily selected as a data source node for data splitting, but in order to avoid erroneous operations such as accidental deletion and modification of data in the original database cluster during the splitting and copying process, a read-only node is generally selected as a real-time data source node of the data splitting method of this embodiment. For example, the second slave node 4032 and the fourth slave node 4042 are both read-only slave nodes, then one of the two can be selected as the data source node, or both can be used as data source nodes.

[0136] Of course, it is understandable that after adding the replication verification mechanism, the possibility of erroneous operation is greatly reduced, and the master node can also be selected as the data source node. Alternatively, for some non-important data, such as independent node 401, which can store some non-important process data, it can also be directly used as the data source node.

[0137] In general, there can be more than one data source node, so that the data in the database cluster to be split can be classified and split, and multiple splitting intermediate media interfaces can be provided to speed up the entire splitting process.

[0138] S202: Send the data to be split in the database cluster to be split to the data source node.

[0139] In this step, the data to be split in the database cluster to be split is copied to the data source node. If it is a complete replacement split, that is, the database cluster to be split is completely replaced by the new database cluster, then all the data in the database cluster to be split is copied to the data source node. If it is a partial replacement split, that is, some services are still retained in the database cluster to be split, then the data to be split corresponding to the migrated services is copied to the data source node.

[0140] It is understandable that, limited by the capacity and processing speed of the data source node, the data to be split can also be sent to the data source node in batches. After the previous batch of data is split to the new database cluster through the data source node, the next batch of data to be split is copied to the data source node.

[0141] S203: Split the data to be split in the data source node according to a preset splitting method to determine split data.

[0142] In this step, the preset splitting method retains the logical structure of the original data table, that is, keeps the table partitioning logic of the database cluster to be split unchanged, and keeps all the data tables in the data to be split as a whole with an unchanged logical structure in the form of virtual tables.

[0143] It is understandable that the business side can still add, delete, read, modify and other operations on the data table using the same database access code, namely SQL code.

[0144] In this embodiment, specifically, as Figure 1a-1c The splitting method for constructing subsequences of the first logical sequence 101 and / or the second logical sequence 102 is shown.

[0145] Figure 5 This is a schematic diagram of constructing a subsequence from a logical sequence according to an embodiment of the present application. Figure 5 As shown, a logic sequence 500 is a continuous or discontinuous sequence of values ​​between a starting value 501 and an ending value 502, for example, the starting value 501 is [0X00] and the ending value 502 is [0XFF]. Between the starting value 501 and the ending value 502, at least one subsequence split value 503, such as [0X80], is taken according to a preset algorithm such as a hash algorithm, then the logic sequence 500 is split into two subsequences, namely, a first subsequence from the starting value 501 to the subsequence split value 503, and a second subsequence from the subsequence split value 503 to the ending value 502.

[0146] It is understandable that the subsequence can be expanded after splitting to correspond to the newly added data records in the data table, and the subsequence can continue to be split in the same way, thereby achieving the universality or reusability of the data splitting code.

[0147] Furthermore, in a possible design, the split data may be determined according to preset splitting parameters corresponding to preset key fields in the data to be split.

[0148] For example, the second logical sequence 102 may be all data in the original data table or part of the data, so the start value, end value and subsequence split value of the second logical sequence need to be adjusted accordingly to obtain the subsequence, i.e., split data.

[0149] S204: Send the split data to the target node of the target database cluster through the data source node.

[0150] In this step, the target database cluster includes at least one new database cluster, each database cluster includes at least one node, and the target node can be any one or more nodes in the new database cluster.

[0151] The split data obtained in step S203 includes not only the data in the original data table in the data to be split, but also a logical sequence to maintain the split table logic of the data table unchanged. Different split data are put into different new databases or nodes of the new database cluster according to preset rules.

[0152] It should be noted that the new database or new database cluster can improve processing efficiency when accessed by the business side because it needs to serve less data, thereby improving the performance of the entire system. Of course, the new database or new database cluster can also be a database with better data processing performance, which can better and faster handle the access needs of the business side when the amount of data is the same.

[0153] Furthermore, in a possible design, after the split data is copied to the target node of the target database cluster through the data source node, the following is further included:

[0154] Verify that the data in the target node has been copied correctly

[0155] If yes, switching part or all of the services of the database cluster to be split to the target database cluster;

[0156] If not, the corresponding problematic data is corrected in the database cluster to be split, and the problematic data is re-copied through the data source node.

[0157] This embodiment provides a data splitting method, which is to determine at least one data source node in the database cluster to be split according to preset requirements; then send the data to be split in the database cluster to be split to the data source node; and split the data to be split in the data source node according to the preset splitting method to determine the split data; finally, send the split data to at least one target node of at least one target database cluster through the data source node. It solves the technical problems in the prior art that data splitting involves many aspects of changes, the splitting code has low reusability, and changing the table splitting logic also requires modifying the query code on the business side. It achieves the technical effect of efficient data splitting without changing the table splitting logic, high splitting code reusability, and no need to change the query code on the business side.

[0158] For ease of understanding, the following embodiments are Figure 2 Based on the illustrated embodiment, several possible implementation methods corresponding to step S202 are explained.

[0159] Figure 6 For the embodiment of this application Figure 2 A schematic diagram of the implementation process of S202 in the data splitting method is shown.

[0160] For S202, sending the data to be split in the database cluster to be split to the data source node, there may be at least the following three possible parallel implementation methods: S610, S602-S603 and S630.

[0161] The first implementation method is:

[0162] S601. According to the attribute characteristics of the data to be split, first data to be split that meets preset conditions is sent to a data source node in a preset sending mode, and the data to be split includes the first data to be split.

[0163] The attribute feature may have multiple features, including: data access frequency, data type, corresponding business classification, etc., and may also include data records of different fields in each data table selecting different value ranges as preset conditions corresponding to the preset attribute feature. Those skilled in the art may select attribute features and their corresponding preset conditions to be met according to the needs of specific application scenarios to filter out the data to be split, which is not limited in this embodiment.

[0164] Furthermore, the first data to be split includes: second data to be split and third data to be split, the attribute characteristics of the second data to be split are greater than or equal to a preset feature threshold, and the attribute characteristics of the third data to be split are less than the preset feature threshold.

[0165] The second data to be split and the third data to be split are sent to the data source node in a preset order.

[0166] Specifically, firstly, one of the second data to be split and the third data to be split is sent to the data source node;

[0167] After sending the split data corresponding to the second data to be split or the third data to be split to the target node through the data source node, the other one is sent to the data source node.

[0168] For example, in the database cluster to be split, data with a data access frequency greater than or equal to the preset frequency is determined as the second data to be split, that is, hot data. Since this part of data is frequently accessed, it has a greater impact on the business side, so it needs to be split into a new database cluster first. Data with a data access frequency less than the preset frequency is determined as the third data to be split, that is, unpopular data. After the second data to be split is split in the data source node and correspondingly split and copied to the target nodes of each target database cluster, the third data to be split can be sent to the data source node for splitting and copying to each target node in its corresponding target database cluster.

[0169] The second implementation method is:

[0170] S602: Arrange the data to be split according to the requirements of preset attribute characteristics to determine a queue of data to be split.

[0171] S603 sends the data to be split to the data source node in batches according to the arrangement order of the data queue to be split.

[0172] In this step, after the previous batch of data is split in the data source node and the split data is sent to the target node through the data source node, the next batch of data is sent to the data source node.

[0173] For example, the data to be split is classified according to the business type, and then arranged according to the operation time recorded in the operation log of the data in each business type. The data in the same time period is sent to the data source node as the same batch of data to be split. The data source node splits it according to the splitting parameters such as the subsequence splitting value, the starting value, and the ending value, and then sends it to each target node in each target database cluster through multiple parallel coroutines. After a batch of data to be split is split and copied, the next batch will be carried out until all the data to be split is split and copied through each data source node.

[0174] The third implementation method is:

[0175] When the number of the data source nodes is at least two, sending the data to be split in the database cluster to be split to the data source nodes includes:

[0176] S604: Distribute the data to be split to different data source nodes according to attribute characteristics of the data to be split.

[0177] In this step, the attribute features include: a first attribute feature and a second attribute feature, the data to be split include: fourth data to be split and fifth data to be split, and the data source nodes include: a first data source node and a second data source node.

[0178] The specific steps include:

[0179] S6041. Send fourth data to be split to the first data source node, wherein a first attribute feature of the fourth data to be split meets a first feature requirement;

[0180] S6042. Send the fifth data to be split to the second data source node, wherein the second attribute feature of the fifth data to be split meets the second feature requirement.

[0181] For example, in the database to be split, among the relevant data of the digital product business, the sales data with a monthly sales amount of more than 100,000 is split into the A database cluster, and the sales data is sent to the first data source node for splitting and replication; among the relevant data of the catering service business, the relevant data with an access frequency greater than or equal to the preset access threshold, such as an average daily visit volume of 2,000 people, is split into the B database cluster, and it is sent to the second data source node for splitting and replication.

[0182] Furthermore, considering that the data source node is also responsible for certain businesses in the database to be split, such as providing complex query, data analysis and other services to the outside world, at this time, there will be data interaction between the data source node and other nodes. Considering that this interaction may affect the splitting and replication, or in order to ensure that the data source node achieves static splitting and static replication, it is necessary to disconnect from other nodes after the data source node receives the data to be split, and stop external services, that is, remove it from the database cluster to be split, and its working status becomes unavailable.

[0183] Below is Figure 2 On the basis of the illustrated embodiment, another data splitting method obtained by removing the data source node is introduced.

[0184] Figure 7 A flowchart of another data splitting method provided for the implementation of this application. Figure 7 As shown, the specific steps of the data splitting method include:

[0185] S701. In a database cluster to be split, determine at least one data source node according to preset requirements.

[0186] In this step, the database cluster to be split includes at least one master-slave relationship node, and the master-slave relationship node includes: a master node and at least one slave node attached to the master node, and the data source node includes at least one of the slave nodes in the master-slave relationship node.

[0187] The preset requirement includes: the data source node is a read-only node, and further, the read-only node is a read-only slave node in the master-slave relationship node.

[0188] For details on this step, see Figure 2 The corresponding S201 of the embodiment is not described in detail here.

[0189] S702: Send the data to be split in the database cluster to be split to the data source node.

[0190] For a detailed explanation of this step, see Figure 6 The illustrated implementation manner will not be described in detail here.

[0191] S703: Remove the data source node from the database cluster to be split.

[0192] In this step, the connection between the data source node and other nodes in the database cluster to be split is cut off, and the type of the data source node is changed to an unavailable type, such as being marked with a NotServing flag. In this way, the external business side cannot access the database through the data source node. This realizes static splitting and static replication for the subsequent splitting and replication process, avoiding errors in the split or replicated data.

[0193] S704: Keep the header structure of the data table unchanged, and determine the split data according to the split interval range and split value corresponding to the preset key field in the data table.

[0194] In this step, the split data has the same structure as the data table, and the data to be split includes at least one data table.

[0195] S705: Send the split data to at least one target node of at least one target database cluster through the data source node.

[0196] For detailed explanation of steps S704-S705, see Figure 2 Steps S203 - S204 of the illustrated embodiment are not described in detail here.

[0197] It should be noted that in a possible design, copying or cloning data to each target node is done in parallel, and the copying method can be set by specifying parameters. For example, multiple coroutines are set to read data from the unavailable source node at the same time and write it to each write queue. By default, there are 10 parallel coroutines that read data from the source node in parallel and write it to the corresponding target node.

[0198] It should also be noted that in one possible situation, if the target node has dirty data, that is, data whose correctness and legality cannot be determined, the dirty data will be deleted or corrected during the process of cloning or copying data to the target node; conversely, it is not allowed to insert data into the target node before cloning the data, and the data will be considered dirty data and deleted.

[0199] S706: Reconnect the data source node to the database cluster to be split.

[0200] In this step, since the business side may write or update data to the database to be split while the data source node is executing steps S704 and S705, in order to ensure that the data in each target node in each target database cluster after the split remains consistent with the data in the database cluster to be split, the data source node needs to be reconnected to the database to be split for inspection.

[0201] In addition, in a possible design, if the data to be split in the database cluster to be split is transmitted in batches, this step can also ensure that the data source node can receive the next batch of data.

[0202] Furthermore, because the data source node also has service functions, in order to shorten the duration of the unavailable state of the data source node as much as possible, the amount of data to be split by the data source node each time and / or the amount of data to be copied to each target node can be limited. By completing the splitting and copying in batches multiple times when the service is idle, the impact on the business side when the database is split can be minimized.

[0203] S707: Send the updated data to be split to the data source node.

[0204] This step can be regarded as repeating step S702, but only the updated data is transmitted to the data source node.

[0205] S708: Split the data to be split again according to a preset splitting method to determine new split data.

[0206] S709: Send the new split data to the target node through the data source node in a filtered replication manner.

[0207] In this step, the filtering and copying method is used to filter out the split data that already exists in the target node.

[0208] Specifically, filtered replication is to start a log consumption service on the target node. The log consumption service goes to the source node to read the binlog, which is used to represent the operation record of the business side on the data when accessing the database. It decides whether to execute the binlog based on the binlog and splitting parameters (such as the initial value, end value and subsequence split value of the logical sequence).

[0209] It should be noted that after the cloning work is completed, the filtered replication is still enabled, that is, ignoring the replication delay, the data of the source database cluster (the database cluster to be split) and the target database cluster are consistent. Now any update operation of the source database cluster can be read on the corresponding target database cluster.

[0210] S710: Check whether the data in the target node has been correctly copied.

[0211] In this step, the data in the data table of the target node may be compared row by row to see whether they are consistent with the data in the data to be split. If so, step S711 is executed; if not, step S712 is executed.

[0212] S711. Switch part or all of the services of the database cluster to be split to the target database cluster.

[0213] In this step, when it is confirmed that the service of the target database cluster is normal, the database cluster to be split can be closed or some nodes of the database cluster to be split can be closed, and the corresponding service can be switched to the target database cluster, that is, the newly split database cluster, and the splitting task is completed.

[0214] S712. Correct the corresponding problem data.

[0215] Specifically, if it is found that there are problems with the data in the database cluster to be split, such as type error or corresponding business type error, the corresponding problematic data is corrected in the database cluster to be split, and the correct data is re-copied through the data source node.

[0216] If an error occurs during the split replication process, the correct data is directly re-replicated through the data source node.

[0217] This embodiment provides a data splitting method, which is to determine at least one data source node in the database cluster to be split according to preset requirements; then send the data to be split in the database cluster to be split to the data source node; and split the data to be split in the data source node according to the preset splitting method to determine the split data; finally, send the split data to at least one target node of at least one target database cluster through the data source node. It solves the technical problems in the prior art that data splitting involves many aspects of changes, the splitting code has low reusability, and changing the table splitting logic also requires modifying the query code on the business side. It achieves the technical effect of efficient data splitting without changing the table splitting logic, high splitting code reusability, and no need to change the query code on the business side.

[0218] Figure 8 The data splitting device 800 is a schematic diagram of a data splitting device provided in an embodiment of the present application. The data splitting device 800 can be implemented by software, hardware or a combination of both.

[0219] like Figure 8 As shown, the data splitting device 800 includes:

[0220] The acquisition module 801 is used to acquire first tag information corresponding to the item to be stored;

[0221] A source node selection module 802 is used to determine at least one data source node in the database cluster to be split according to preset requirements, wherein the data source node is an intermediate medium for data replication and transfer between the database cluster to be split and the target database cluster;

[0222] The to-be-split data preparation module 803 is used to send the to-be-split data in the to-be-split database cluster to the data source node, and the preset splitting method retains the logical structure of the original data table;

[0223] A splitting module 804 is used to split the data to be split in the data source node according to a preset splitting method to determine split data;

[0224] The splitting module 804 is further configured to send the split data to at least one target node of at least one target database cluster through the data source node.

[0225] In one possible design, the database cluster to be split includes at least one master-slave relationship node, and the master-slave relationship node includes: a master node and at least one slave node attached to the master node, and the data source node includes at least one of the slave nodes in the master-slave relationship node.

[0226] In a possible design, the splitting module 804 is specifically used to determine the split data according to preset splitting parameters corresponding to preset key fields in the data to be split.

[0227] Optionally, the preset splitting parameter includes: a splitting interval range and at least one splitting value within the splitting interval range, and the splitting module 804 is specifically configured to:

[0228] Keeping the structure of the data table unchanged, the split data is determined according to the split interval range and the split value corresponding to the preset key field in the data table, the split data has the same structure as the data table, and the data to be split includes at least one of the data tables.

[0229] In a possible design, the splitting module 804 is further specifically configured to:

[0230] Creating a write queue for each of the target nodes;

[0231] Using a read coroutine to read the split data from the data source node, and inserting the split data into the write queue;

[0232] The split data is copied from the write queue to the corresponding target node in sequence using a write coroutine.

[0233] In a possible design, the splitting module 804 is further used to remove the data source node from the database cluster to be split, so that the total amount of data in the data source node remains unchanged.

[0234] In a possible design, the splitting module 804 is further used to:

[0235] Reconnecting the data source node to the database cluster to be split;

[0236] Sending the updated data to be split to the data source node;

[0237] Splitting the data to be split again according to a preset splitting method to determine new split data;

[0238] The new split data is sent to the target node through the data source node in a filtering and copying manner, and the filtering and copying manner is used to filter out the split data that already exists in the target node.

[0239] Optionally, the preset requirement includes: the data source node is a read-only node.

[0240] Further optionally, the read-only node is a read-only slave node in the master-slave relationship node.

[0241] In one possible design, the data to be split preparation module 803 is used to send the first data to be split that meets preset conditions to the data source node in a preset sending manner according to the attribute characteristics of the data to be split, and the data to be split includes the first data to be split.

[0242] Optionally, the to-be-split data preparation module 803 is specifically used to:

[0243] Sending the second data to be split and the third data to be split to the data source node in a preset order, wherein the attribute feature of the second data to be split is greater than or equal to a preset feature threshold, and the attribute feature of the third data to be split is less than the preset feature threshold;

[0244] The first data to be split includes the second data to be split and the third data to be split.

[0245] Optionally, the to-be-split data preparation module 803 is specifically used to:

[0246] First, sending one of the second data to be split and the third data to be split to the data source node;

[0247] After sending the split data corresponding to the second data to be split or the third data to be split to the target node through the data source node, the other one is sent to the data source node.

[0248] In a possible design, the to-be-split data preparation module 803 is used to arrange the to-be-split data according to the requirements of preset attribute characteristics to determine a to-be-split data queue;

[0249] The to-be-split data preparation module 803 is further configured to send the to-be-split data to the data source node in batches according to the arrangement order of the to-be-split data queue; wherein:

[0250] After the previous batch of data is split in the data source node and the split data is sent to the target node through the data source node, the next batch of data is sent to the data source node.

[0251] In a possible design, when the number of the data source nodes is at least two, the data to be split preparation module 803 is used to distribute the data to be split to different data source nodes according to attribute characteristics of the data to be split.

[0252] Optionally, the attribute feature includes: a first attribute feature and a second attribute feature, and the to-be-split data preparation module 803 is used to send the fourth to-be-split data to the first data source node, and the first attribute feature of the fourth to-be-split data meets the first feature requirement;

[0253] The to-be-split data preparation module 803 is further configured to send fifth to-be-split data to the second data source node, wherein the second attribute feature of the fifth to-be-split data meets the second feature requirement;

[0254] The data to be split include: the fourth data to be split and the fifth data to be split, and the data source nodes include: the first data source node and the second data source node.

[0255] In a possible design, the data splitting device further includes:

[0256] Verification module 805, used to verify whether the data in the target node has been correctly copied;

[0257] If so, then

[0258] A switching module 806 is used to switch part or all of the services of the database cluster to be split to the target database cluster;

[0259] If not, then

[0260] The splitting module 804 is also used to correct the corresponding problem data.

[0261] It is worth mentioning that Figure 8 The device provided in the illustrated embodiment can execute the method provided in any of the above method embodiments. Its specific implementation principles, technical features, professional terminology explanations and technical effects are similar and will not be repeated here.

[0262] Fig. 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Fig. 9 As shown, the electronic device 900 may include: at least one processor 901 and a memory 902 . Fig. 9 An electronic device is shown using a processor as an example.

[0263] The memory 902 is used to store programs. Specifically, the programs may include program codes, and the program codes include computer operation instructions.

[0264] The memory 902 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0265] The processor 901 is used to execute the computer-executable instructions stored in the memory 902 to implement the methods described in the above method embodiments.

[0266] The processor 901 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0267] Optionally, the memory 902 may be independent or integrated with the processor 901. When the memory 902 is a device independent of the processor 901, the electronic device 900 may further include:

[0268] The bus 903 is used to connect the processor 901 and the memory 902. The bus may be an industry standard architecture (ISA) bus, a peripheral component interconnect (PCI) bus, or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc., but it does not mean that there is only one bus or one type of bus.

[0269] Optionally, in a specific implementation, if the memory 902 and the processor 901 are integrated on a chip, the memory 902 and the processor 901 can communicate through an internal interface.

[0270] An embodiment of the present application also provides a computer-readable storage medium, which may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, and other media that can store program codes. Specifically, the computer-readable storage medium stores program instructions, and the program instructions are used for the methods in the above-mentioned method embodiments.

[0271] An embodiment of the present application further provides a computer program product, including a computer program, which implements the methods in the above-mentioned method embodiments when executed by a processor.

[0272] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the claims of the present application.

[0273] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

[0274] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A data splitting method, characterized in that: include: In the database cluster to be split, determine at least one data source node according to preset requirements; Sending the data to be split in the database cluster to be split to the data source node; Splitting the data to be split in the data source node according to a preset splitting method to determine split data, wherein the preset splitting method retains the logical structure of the original data table; Sending the split data to a target node of a target database cluster through the data source node; The method of splitting the data to be split in the data source node according to a preset splitting method to determine the split data includes: determining the split data according to preset splitting parameters corresponding to preset key fields in the data to be split; the preset splitting parameters include: a splitting interval range and at least one splitting value within the splitting interval range; the method of determining the split data according to the preset splitting parameters corresponding to the preset key fields in the data to be split includes: keeping the header structure of the data table unchanged, determining the split data according to the splitting interval range and the splitting value corresponding to the preset key fields in the data table, the split data having the same structure as the data table, and the data to be split includes at least one of the data tables.

2. The data splitting method according to claim 1, characterized in that: The database cluster to be split includes at least one master-slave relationship node, which includes: a master node and at least one slave node attached to the master node, and the data source node includes at least one slave node in the master-slave relationship node.

3. The data splitting method according to claim 1, characterized in that: The sending the split data to the target node of the target database cluster through the data source node includes: Creating a write queue for each of the target nodes; Using a read coroutine to read the split data from the data source node, and inserting the split data into the write queue; The split data is copied from the write queue to the corresponding target node in sequence using a write coroutine.

4. The data splitting method according to any one of claims 1 to 3, characterized in that: After sending the data to be split in the database cluster to be split to the data source node, the method further includes: The data source node is removed from the database cluster to be split so that the total amount of data in the data source node remains unchanged.

5. The data splitting method according to claim 4, characterized in that: After sending the split data to at least one target node of at least one target database cluster through the data source node, the method further includes: Reconnecting the data source node to the database cluster to be split; Sending the updated data to be split to the data source node; Splitting the data to be split again according to a preset splitting method to determine new split data; The new split data is sent to the target node through the data source node in a filtering and copying manner, and the filtering and copying manner is used to filter out the split data that already exists in the target node.

6. The data splitting method according to claim 5, characterized in that: The preset requirement includes: the data source node is a read-only node.

7. The data splitting method according to claim 6, characterized in that: The read-only node is a read-only slave node in a master-slave relationship.

8. The data splitting method according to any one of claims 1 to 3, characterized in that: The sending the data to be split in the database cluster to be split to the data source node comprises: According to the attribute characteristics of the data to be split, first data to be split that meets the preset conditions is sent to the data source node in a preset sending manner, and the data to be split includes the first data to be split.

9. The data splitting method according to claim 8, characterized in that: The step of sending the first data to be split that meets a preset condition to the data source node in a preset sending manner according to the attribute characteristics of the data to be split includes: Sending the second data to be split and the third data to be split to the data source node in a preset order, wherein the attribute feature of the second data to be split is greater than or equal to a preset feature threshold, and the attribute feature of the third data to be split is less than the preset feature threshold; The first data to be split includes the second data to be split and the third data to be split.

10. The data splitting method according to claim 9, characterized in that: The sending the second to-be-split data and the third to-be-split data to the data source node in a preset order includes: First, sending one of the second data to be split and the third data to be split to the data source node; After sending the split data corresponding to the second data to be split or the third data to be split to the target node through the data source node, the other one is sent to the data source node.

11. The data splitting method according to any one of claims 1 to 3, characterized in that: The sending the data to be split in the database cluster to be split to the data source node comprises: Arranging the data to be split according to the requirements of preset attribute characteristics to determine a queue of data to be split; According to the arrangement order of the queue of data to be split, the data to be split is sent to the data source node in batches; wherein, After the previous batch of data is split in the data source node and the split data is sent to the target node through the data source node, the next batch of data is sent to the data source node.

12. The data splitting method according to any one of claims 1 to 3, characterized in that: When the number of the data source nodes is at least two, sending the data to be split in the database cluster to be split to the data source nodes includes: The data to be split are distributed to different data source nodes according to attribute characteristics of the data to be split.

13. The data splitting method according to claim 12, characterized in that: The attribute characteristics include: a first attribute characteristic and a second attribute characteristic, and distributing the data to be split to different data source nodes according to the attribute characteristics of the data to be split includes: Sending fourth data to be split to the first data source node, wherein a first attribute feature of the fourth data to be split meets a first feature requirement; Sending fifth data to be split to the second data source node, wherein the second attribute feature of the fifth data to be split meets the second feature requirement; The data to be split include: the fourth data to be split and the fifth data to be split, and the data source nodes include: the first data source node and the second data source node.

14. The data splitting method according to any one of claims 1 to 3, characterized in that: After sending the split data to at least one target node of at least one target database cluster through the data source node, the method further includes: Verify whether the data in the target node has been correctly copied; If yes, switching part or all of the services of the database cluster to be split to the target database cluster; If not, correct the corresponding problem data.

15. A data splitting device, characterized in that: include: A source node selection module, used to determine at least one data source node in the database cluster to be split according to preset requirements; A to-be-split data preparation module, used to send the to-be-split data in the to-be-split database cluster to the data source node; A splitting module, used for splitting the data to be split in the data source node according to a preset splitting method to determine split data, wherein the preset splitting method retains the logical structure of the original data table; The splitting module is further used to send the split data to at least one target node of at least one target database cluster through the data source node; The splitting module is specifically used to: determine the split data according to the preset splitting parameters corresponding to the preset key fields in the data to be split, and the preset splitting parameters include: a splitting interval range and at least one splitting value within the splitting interval range; The splitting module is specifically used to: keep the header structure of the data table unchanged, determine the split data according to the splitting interval range and the splitting value corresponding to the preset key field in the data table, the split data has the same structure as the data table, and the data to be split includes at least one of the data tables.

16. An electronic device, characterized in that: include: processor and memory; wherein, The memory is used to store a computer program of the processor; The processor is configured to perform the data splitting method according to any one of claims 1 to 14 by executing the computer program.

17. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the data splitting method described in any one of claims 1 to 14 is implemented.

18. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the data splitting method described in any one of claims 1 to 14 is implemented.

Images