Data sharding and database splitting method and device, computer device and storage medium

Abstract

This application provides a data sharding and database partitioning method, apparatus, computer equipment, and storage medium applicable to the field of big data technology. The method includes: sharding business data to obtain at least one logical shard; determining the logical shard number of the logical shard based on element strings in the logical shard; intercepting proxy object information corresponding to the logical shard; extracting SQL statements from the proxy object information; obtaining the database name and physical table name corresponding to the logical shard number according to database partitioning rules; setting the database corresponding to the database name in the data cluster as the target database; modifying the logical table name in the SQL statement to the physical table name to obtain a rewritten SQL statement; and executing the rewritten SQL statement in the target database. This application reduces the read / write pressure on each database, eliminates the risk of slow database operation, avoids abnormal database downtime, and facilitates the expansion of the data cluster.

Description

Technical Field

[0001] This application relates to the field of big data technology, and in particular to a data sharding and database partitioning method, apparatus, computer equipment, and storage medium. Background Technology

[0002] A database is a repository for storing data. It has a large storage capacity, capable of holding millions, tens of millions, or even hundreds of millions of data entries. Currently, for massive amounts of business data from a large number of users, a single database is typically used to store this massive transaction data. This massive business data refers to a large amount of internal business transaction information within an enterprise, primarily including online transaction data and online analytical data. It is structured, static, and historical data managed and accessed through a relational database.

[0003] However, the inventors discovered that due to the sheer volume of business data and the limited capacity of a single database, processing massive amounts of business data in a single database would lead to slow operation or even abnormal crashes due to excessive read / write pressure. Summary of the Invention

[0004] This application provides a data sharding and database partitioning method, apparatus, computer equipment, and storage medium to solve the problem that processing massive amounts of business data in a single database can lead to slow operation or even abnormal crashes due to excessive read / write pressure on the single database.

[0005] Firstly, this application provides a data sharding and database partitioning method, including:

[0006] The system receives business data, performs sharding on the business data to obtain at least one logical shard, and determines the logical shard number of the logical shard based on the element strings in the logical shard. The business data is data information used to input into the data cluster, the element strings are used to describe the attribute characteristics of sub-data in the logical shard, and the element strings consist of at least one business element or at least one business time. The business element is metadata used to describe the sub-data, and the business time refers to the generation time of the sub-data.

[0007] Intercept the proxy object information corresponding to the logical shard, and extract the SQL statement in the proxy object information, wherein the proxy object information is used to perform add processing, and / or delete processing, and / or modify processing, and / or query processing on the logical shard;

[0008] According to the preset database sharding and table partitioning rules, the database name and physical table name corresponding to the logical shard number are obtained. The database in the data cluster corresponding to the database name is set as the target database. The logical table name in the SQL statement is modified to the physical table name to obtain the rewritten SQL statement. The logical table name is the data table name indicated in the SQL statement, and the physical table name is the data table name defined by the logical table name in the target database.

[0009] The rewritten SQL statement is executed in the target database to perform add, delete, modify, and / or query processing on the logical shard in the target database.

[0010] In the above scheme, the step of performing sharding processing on the business data to obtain at least one logical shard includes:

[0011] Identify the element type of each business element in the business data, and aggregate at least one business element with the same element type into an element set, wherein the element type represents the category of the business element;

[0012] The column data corresponding to the business elements in each of the element sets is extracted from the business data to obtain the logical fragments corresponding to each of the element sets, wherein the column data is the sub-data corresponding to each business element in the business data.

[0013] In the above scheme, the step of performing sharding processing on the business data to obtain at least one logical shard includes:

[0014] Based on the business time of the row data, at least one row of data in the business data is sorted in ascending or descending order to obtain a business sequence, wherein the row data is sub-data generated at different times in the business data;

[0015] The time span information of the business data is obtained based on the business time at the beginning of the business sequence and the business time at the end of the business sequence.

[0016] The time span information is divided according to the preset number of segments to obtain time periods corresponding to the number of segments;

[0017] From the business time of acquiring each row of data, and summing the business times belonging to the same time period, at least one time set is obtained;

[0018] Extract the row data corresponding to the business time in each of the time sets from the business data to obtain the logical slices corresponding to each of the time sets.

[0019] In the above scheme, determining the logical shard number of the logical shard based on the element string in the logical shard includes:

[0020] Extract at least one business element from the logical segment and concatenate at least one business element to obtain an element string; or extract at least one business time from the logical segment and concatenate at least one business time to obtain an element string.

[0021] The element string is processed according to preset sharding rules to obtain a library shard number and a table shard number. The logical shard number is obtained by concatenating the library shard number and the table shard number.

[0022] The element string is processed according to a preset hash algorithm to obtain a hash value. The hash value is then processed according to preset database and table operation rules to obtain a database shard number and a table shard number. The database shard number and the table shard number are then concatenated to obtain a logical shard number.

[0023] In the above scheme, the interception of the proxy object information corresponding to the logical shard includes:

[0024] If a mapping interface is detected to be called, the proxy object information received in the mapping interface is intercepted and the proxy fragment number in the proxy object information is extracted. The mapping interface is a computer interface used to process business data corresponding to the proxy object information based on the proxy object information.

[0025] The proxy object information whose proxy shard number matches the logical shard number of the logical shard is set as the proxy object information corresponding to the logical shard.

[0026] In the above scheme, obtaining the database name and physical table name corresponding to the logical shard number according to the preset database sharding and table partitioning rules includes:

[0027] Extract the database shard number from the logical shard number, and obtain the database name that matches the database shard number from the database sharding and table partitioning rules;

[0028] Extract the table shard number from the logical shard number, and obtain the physical table name corresponding to the table shard number from at least one physical table name corresponding to the database name in the database sharding and table partitioning rules.

[0029] In the above scheme, setting the database in the data cluster corresponding to the database name as the target database includes:

[0030] Obtain a data list from the data cluster, wherein the data list contains the source name of at least one database;

[0031] Iterate through the data list and set the database whose source name matches the database name as the target database;

[0032] Establish a communication connection with the target database, the communication connection being a communication line used to access the target database.

[0033] In the above scheme, modifying the logical table name in the SQL statement to the physical table name to obtain the rewritten SQL statement includes:

[0034] The SQL statement is parsed to obtain an abstract syntax tree with at least one instruction vocabulary. The parsing refers to performing syntactic analysis and / or semantic analysis on the SQL statement to obtain statement logic, and decomposing the SQL statement according to the statement logic to obtain computer rules with at least one instruction having a specific meaning.

[0035] Identify the logical table name in the abstract syntax tree, modify the logical table name to the physical table name, and generate a rewritten SQL statement based on the abstract syntax tree with the physical table name.

[0036] In the above scheme, the step of identifying the logical table name in the abstract syntax tree, modifying the logical table name to the physical table name, and generating a rewritten SQL statement based on the abstract syntax tree containing the physical table name includes:

[0037] Each node in the abstract syntax tree is identified by a pre-defined table name regular expression, and the node containing the logical table name is obtained. The table name regular expression contains keywords that represent the logical table name.

[0038] The logical table name is modified to the physical table name to obtain an abstract syntax tree with the physical table name. Based on the syntax logic of the abstract syntax tree, the abstract syntax tree with the physical table name is converted into a rewritten SQL statement.

[0039] In the above scheme, before receiving the service data, the method further includes:

[0040] Obtain at least one database, and name each database to obtain at least one source name;

[0041] Create a data center server, and connect at least one of the databases to the data center server to obtain a data cluster;

[0042] The source names of each database are compiled to form a data list, which is then entered into the configuration file of the data center server.

[0043] In the above scheme, after entering the data list into the configuration file of the data center server, the method further includes:

[0044] Obtain the configuration file, which contains at least one of the database names and rule information for allocating logical shards to the databases corresponding to each of the database names;

[0045] The configuration file is parsed to obtain the database sharding and table partitioning rules, which are executable files containing the rule information.

[0046] Secondly, this application provides a data sharding and database partitioning device, comprising:

[0047] A data sharding module is used to receive business data, shard the business data to obtain at least one logical shard, and determine the logical shard number of the logical shard based on the element strings in the logical shard; wherein, the business data is data information used to input into the data cluster, and the business elements are used to characterize the attribute features of the logical shard;

[0048] The input interception module is used to intercept the proxy object information corresponding to the logical shard and extract the SQL statements in the proxy object information, wherein the proxy object information is used to perform addition processing, and / or deletion processing, and / or modification processing, and / or query processing on the logical shard;

[0049] The database sharding and table partitioning module is used to obtain the database name and physical table name corresponding to the logical shard number according to the preset database sharding and table partitioning rules, set the database in the data cluster corresponding to the database name as the target database, and modify the logical table name in the SQL statement to the physical table name to obtain the rewritten SQL statement. The logical table name is the data table name indicated in the SQL statement, and the physical table name is the data table name defined by the logical table name in the target database.

[0050] The statement execution module is used to execute the rewritten SQL statement in the target database to perform add, delete, modify, and / or query processing on the logical shard in the target database.

[0051] Thirdly, this application provides a computer device, including: a processor, and a memory communicatively connected to the processor;

[0052] The memory stores computer-executed instructions;

[0053] The processor executes computer execution instructions stored in the memory to implement the above-described method.

[0054] Fourthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the above-described method.

[0055] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0056] This application provides a data sharding and database partitioning method, apparatus, computer equipment, and storage medium. By sharding business data, it achieves the effect of reducing the size of business data and obtaining logical sharding. By determining the logical shard number of each logical shard, it facilitates the storage of each logical shard in the corresponding database and data table, thus providing a foundation for distributed data storage.

[0057] By intercepting the proxy object information corresponding to the logical shard, it is possible to perform add, delete, modify, and query operations in the database.

[0058] By setting the database corresponding to the database name in the data cluster as the target database, the database that should store the logical shard is locked; by modifying the logical table name in the SQL statement to the physical table name to obtain the rewritten SQL statement, the physical table for data processing in the SQL statement, i.e., the data table, is accurately obtained. At the same time, by setting the logical table name in the SQL statement, the client can generate SQL statements according to their usage habits and logic, which provides convenience for the client.

[0059] By executing the rewritten SQL statements in the target database to perform add, delete, modify, and / or query processing on the logical shards in the target database, the effect of using multiple databases to process massive business data is achieved, reducing the read and write pressure of each database, eliminating the risk of slow database operation efficiency, and avoiding abnormal database downtime.

[0060] Meanwhile, when expansion is required in the data cluster, it is only necessary to modify the correspondence between logical shard numbers and database names and physical table names in the database sharding rules. This establishes a mapping relationship between some logical shard numbers in the database sharding rules and the newly added databases in the data cluster. This allows the business data corresponding to some logical shard numbers in the original databases in the data cluster to be migrated to the newly added databases, greatly reducing the amount of data migration during data cluster expansion and providing great convenience for data cluster expansion. Attached Figure Description

[0061] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0062] Figure 1 This is a schematic diagram illustrating an application scenario of a data sharding and database partitioning method according to this application;

[0063] Figure 2 This is a flowchart of Embodiment 1 of a data fragmentation and database partitioning method according to this application;

[0064] Figure 3 This is a flowchart of Embodiment 2 of a data fragmentation and database partitioning method according to this application;

[0065] Figure 4 This is a schematic diagram of the structure of Embodiment 3 of a data sharding and database partitioning device according to this application;

[0066] Figure 5 This is a schematic diagram of the hardware structure of the computer device in the computer device of the present invention.

[0067] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0068] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0069] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with relevant laws, regulations and standards, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0070] It should be noted that the data sharding and database partitioning method, apparatus, computer equipment and storage medium of this application can be used in the field of big data technology, or in any field other than big data technology. The application field of the data sharding and database partitioning method, apparatus, computer equipment and storage medium of this application is not limited.

[0071] Please see Figure 1 The specific application scenario of this application is:

[0072] A server 11 with a data sharding and database partitioning method is running. A data cluster 12 is connected. The data cluster 12 has at least one database 121. The server 11 receives business data, performs sharding processing on the business data to obtain at least one logical shard, and determines the logical shard number of the logical shard based on the element string in the logical shard.

[0073] Server 11 intercepts the proxy object information corresponding to the logical shard and extracts the SQL statements in the proxy object information. The proxy object information is used to perform addition, deletion, modification, and query processing on the logical shard.

[0074] Server 11 obtains the database name and physical table name corresponding to the logical shard number according to the preset database sharding and table partitioning rules, sets the database 121 corresponding to the database name in the data cluster 12 as the target database 122, and modifies the logical table name in the SQL statement to the physical table name to obtain the rewritten SQL statement.

[0075] Server 11 executes rewritten SQL statements in target database 122 to perform add, delete, modify, and / or query processing on logical shards in target database 122.

[0076] The data sharding and database partitioning method provided in this application allows for the sharing of a client's risk assessment information across the entire consortium of financial institutions, once the client completes a risk assessment. Furthermore, the application of big data technology ensures the immutability of the client's risk assessment, guaranteeing information accuracy and improving risk control efficiency. Due to the use of encryption technology, client information is not easily leaked or cracked. This approach aims to solve the aforementioned technical problems of existing technologies.

[0077] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0078] Example 1:

[0079] Please see Figure 2 This application provides a data sharding and database partitioning method, including:

[0080] S201: Receive business data, perform sharding on the business data to obtain at least one logical shard, and determine the logical shard number of the logical shard based on the element string in the logical shard; wherein, business data is the data information used to input into the data cluster, the element string is used to describe the attribute characteristics of the sub-data in the logical shard, the element string consists of at least one business element or at least one business time, the business element is the metadata used to describe the sub-data, and the business time refers to the generation time of the sub-data.

[0081] In this step, the business data is sharded to reduce its size and achieve logical sharding. By determining the logical shard number of each logical shard, each logical shard can be stored in the corresponding database and table, thus providing a foundation for distributed data storage.

[0082] In a preferred embodiment, the business data is fragmented to obtain at least one logical fragment, including:

[0083] Identify the element type of each business element in the business data, and aggregate at least one business element with the same element type into an element set, where the element type represents the category of the business element.

[0084] Extract the column data corresponding to the business elements in each element set from the business data to obtain the logical shards corresponding to each element set. The column data is the sub-data corresponding to each business element in the business data.

[0085] For example, business elements are metadata describing each sub-data item in business data, and element types are metadata for business elements. For instance, element types might include: basic user information, user transaction information, and user relationship information; business elements might include: name, gender, age; stock trading, bond trading, and deposit trading; investment companies and employing companies. Therefore, the element type for name, gender, and age is basic user information; the element type for stock trading, bond trading, and deposit trading is user transaction information; and the element type for investment companies and employing companies is user relationship information. Thus, extracting the column data for name, gender, and age under basic user information in the business data constitutes one logical slice; extracting the column data for stock trading, bond trading, and deposit trading under basic user information in the business data constitutes another logical slice; and extracting the column data for investment companies and employing companies under basic user information in the business data constitutes yet another logical slice.

[0086] Optionally, business elements can also be the numbers of various metadata in the business data, such as id1, id2, id3, id4; it can also be determined whether each business element belongs to the same element type according to a preset calculation rule. For example, if the calculation rule is id%2, then business elements id1 and id3 are of the same element type, and id2 and id4 are of the same element type. Therefore, the data corresponding to id1 and id3 in the business data will be split into one logical shard, and the data corresponding to id2 and id4 in the business data will be split into another logical shard. In a preferred embodiment, the business data is sharded to obtain at least one logical shard, including:

[0087] Based on the business time of the row data, at least one row of data in the business data is sorted in ascending or descending order to obtain the business sequence, where the row data is sub-data generated at different times in the business data;

[0088] Based on the business time at the beginning of the business sequence and the business time at the end of the business sequence, the time span information of the business data is obtained;

[0089] The time span information is divided according to the preset number of segments to obtain the time period corresponding to the number of segments;

[0090] The business time of each row of data is obtained, and the business times belonging to the same time period are summarized to obtain at least one time set;

[0091] Extract the row data corresponding to the business time in each time set from the business data to obtain the logical shards corresponding to each time set.

[0092] For example, suppose the row data in the business data includes: row data 1-9:00, row data 2-10:30, row data 3-11:20, row data 4-12:40, row data 5-13:10, row data 6-14:50, row data 7-15:30, row data 8-16:10, row data 9-17:00, and row data 10-18:00. Therefore, the obtained time span information is 9 hours. If the number of shards is 3, then the following time periods will be obtained: [9:00, 12:00], [12:01, 15:00], [15:01, 18:00]. Therefore, row data 1-9:00, row data 2-10:30, and row data 3-11:20 will be set as one logical shard, row data 4-12:40, row data 5-13:10, and row data 6-14:50 will be set as another logical shard, and row data 7-15:30, row data 8-16:10, row data 9-17:00, and row data 10-18:00 will be set as yet another logical shard.

[0093] In a preferred embodiment, determining the logical shard number of a logical shard based on the element string in the logical shard includes:

[0094] Extract at least one business element from the logical slice and concatenate the at least one business element to obtain an element string; or extract at least one business time from the logical slice and concatenate the at least one business time to obtain an element string.

[0095] The element string is processed according to the preset sharding rules to obtain the database shard number and table shard number. The logical shard number is obtained by concatenating the database shard number and table shard number.

[0096] The element string is processed according to a preset hash algorithm to obtain a hash value. The hash value is then processed according to preset database and table operation rules to obtain a database shard number and a table shard number. The database shard number and the table shard number are then concatenated to obtain a logical shard number.

[0097] For example, if a logical shard is obtained by extracting different business elements, such as: the business elements in logical shard 1 include: name, gender and age; concatenate name, gender and age to obtain the binary value corresponding to the element string name gender age, or the encoding corresponding to name gender age, such as: 101102103.

[0098] If a logical shard is obtained by extracting different business times, for example, the business times in logical shard 2 include 9:00, 10:30 and 11:20, then concatenating 9:00, 10:30 and 11:20 will yield the element string 090010301120.

[0099] Sharding rules include database rules and table rules.

[0100] The library rules include: adding each number in the element string to get the library sum, taking the remainder by dividing the library sum by the library value to get the library remainder, and using the library remainder as the library segment number; the library value can be set to any positive integer.

[0101] The table rules include: adding each number in the element string to get the table sum, taking the remainder by dividing the table sum by the table value, and using the remainder as the table shard number; the table value can be set to any positive integer.

[0102] For example: if the database shard number is 1 and the table shard number is 3 according to the sharding rules, then logical shard numbers 1-3 are obtained, which indicate that the logical shards are saved to data table 3 in database 1.

[0103] A hash value is obtained by performing a hash operation on the binary values ​​corresponding to a name, gender, and age, or the codes corresponding to those names, genders, and ages. A hash algorithm maps a binary string of arbitrary length to a binary string of fixed length; the rule governing this mapping is the hash algorithm, and the resulting binary string after mapping the original data is the hash value.

[0104] The rules for database and table operations include database rules and table rules.

[0105] The library rules include: taking the remainder of the hash value with the library value as the dividend, and using the remainder as the library shard number; the library value can be set to any positive integer.

[0106] The table rules include: taking the remainder of the hash value with the table value as the dividend, and using the remainder as the table shard number; the table value can be set to any positive integer.

[0107] S202: Intercept the proxy object information corresponding to the logical shard, and extract the SQL statements in the proxy object information. The proxy object information is used to perform add, delete, modify, and query processing on the logical shard.

[0108] In this step, the proxy object information records the methods for processing logical shards, such as SQL statements, thereby enabling the addition, deletion, modification, and query processing of logical shards in the database.

[0109] The Mapper interface intercepts proxy object information corresponding to logical shards. The Mapper interface is a proxy class generated using JDK dynamic proxies, which contains mapping files to facilitate operations such as adding, deleting, modifying, and querying data in the database.

[0110] In a preferred embodiment, intercepting proxy object information corresponding to logical shards includes:

[0111] If the mapping interface is detected to be called, the proxy object information received in the mapping interface is intercepted and the proxy fragment number in the proxy object information is extracted. The mapping interface is a computer interface used to process the business data corresponding to the proxy object information according to the proxy object information.

[0112] The proxy object information whose proxy shard number matches the logical shard number of the logical shard is set as the proxy object information corresponding to the logical shard.

[0113] For example, the Mapper interface is used as the mapping interface. When this interface is called, it is used to perform operations in the database. However, it is currently impossible to know which logical shard the proxy object information corresponds to. Therefore, by intercepting the proxy object information received in the mapping interface and extracting the proxy shard number from the proxy object information, the correspondence between logical shards and operation processing can be determined.

[0114] S203: Based on the preset database sharding and table partitioning rules, obtain the database name and physical table name corresponding to the logical shard number, set the database corresponding to the database name in the data cluster as the target database, and modify the logical table name in the SQL statement to the physical table name to obtain the rewritten SQL statement. Here, the logical table name is the data table name indicated in the SQL statement, and the physical table name is the data table name defined by the logical table name in the target database.

[0115] In this step, the logical table name is the data table name indicated in the SQL statement, and the physical table name is the data table name defined by the logical table name in the target database.

[0116] The sharding rules record the mapping relationship between logical shard numbers and database and physical table names. The database name and physical table name corresponding to the logical shard number are obtained by following the preset sharding rules.

[0117] By setting the database corresponding to the database name in the data cluster as the target database, the database that should store the logical shard is locked. The rewritten SQL statement is obtained by modifying the logical table name in the SQL statement to the physical table name, thus accurately obtaining the physical table, i.e., the data table, used for data processing in the SQL statement. Simultaneously, by setting the logical table name in the SQL statement, the client can generate SQL statements according to their usage habits and logic, providing convenience for the client.

[0118] A logical table name is the name of a data table indicated in an SQL statement. For example, a logical table name represents a virtual logical table. A logical table can be understood as a view in the database; it is a virtual table. It can be mapped to a single physical table or composed of multiple physical tables from different databases. Logically, it represents the sum of tables with the same logical and data table structure. For example, `t_order` is split into 10 tables: `t_order_0`, `t_order_9`, etc. At this point, the `t_order` table no longer exists in the database, but the SQL in our code is still written according to `t_order`. In this case, `t_order` is the logical table of these split tables.

[0119] A physical table name is the name of a data table defined in the target database as a logical table name. For example, a physical table name represents a real physical table, which is a table that actually exists in a specific database. Examples include t_order_0....t_order_9 mentioned above. A physical table name can also be the storage path of the physical table.

[0120] In a preferred embodiment, the database name and physical table name corresponding to the logical shard number are obtained according to preset database sharding and table partitioning rules, including:

[0121] Extract the database shard number from the logical shard number, and obtain the database name that matches the database shard number from the database sharding and table partitioning rules;

[0122] Extract the table shard number from the logical shard number, and obtain the physical table name corresponding to the table shard number from at least one physical table name corresponding to the database name in the database sharding rules.

[0123] By sequentially obtaining the database name matching the database shard number, and then obtaining the physical table name corresponding to the table shard number under that database name, the efficiency of obtaining the physical table is greatly improved.

[0124] In a preferred embodiment, setting the database corresponding to the database name in the data cluster as the target database includes:

[0125] Retrieve a list of data in the data cluster, which contains the source name of at least one database;

[0126] Iterate through the data list and set the database whose source name matches the database name as the target database;

[0127] Establish a communication connection with the target database. The communication connection is the communication line used to access the target database.

[0128] By establishing a communication link with the target database, the ability to access the target database can be achieved.

[0129] In a preferred embodiment, modifying the logical table name in the SQL statement to the physical table name yields the rewritten SQL statement, including:

[0130] Parsing an SQL statement yields an abstract syntax tree containing at least one instruction vocabulary. Parsing refers to performing syntactic and / or semantic analysis on the SQL statement to obtain statement logic, and then decomposing the SQL statement based on the statement logic to obtain computer rules containing at least one instruction with specific meaning.

[0131] Identify logical table names in the abstract syntax tree, modify the logical table names to physical table names, and generate rewritten SQL statements based on the abstract syntax tree with physical table names.

[0132] For example, performing syntax analysis and / or semantic analysis on an SQL statement yields an abstract syntax tree (AST) with at least one instruction vocabulary. The abstract syntax tree is a tree-like representation of the abstract syntax structure of the SQL statement. Each node in the tree records an instruction vocabulary from the SQL statement. The abstract syntax tree as a whole represents the statement logic of the SQL statement.

[0133] Identify the nodes in the abstract syntax tree that record logical table names, and convert the logical table names in those nodes into physical table names to obtain an abstract syntax tree with physical table names. Based on the syntax logic of the abstract syntax tree, restore each node in the abstract syntax tree to form the rewritten SQL statement.

[0134] Furthermore, logical table names in the abstract syntax tree are identified and modified to physical table names. Rewritten SQL statements are then generated based on the abstract syntax tree containing the physical table names, including:

[0135] Each node in the abstract syntax tree is identified by a pre-defined table name regular expression, and the node containing the logical table name is obtained. The table name regular expression contains keywords that represent the logical table name.

[0136] The logical table name is changed to the physical table name to obtain the abstract syntax tree with the physical table name. Based on the syntax logic of the abstract syntax tree, the abstract syntax tree with the physical table name is converted into rewritten SQL statements.

[0137] For example, keywords in the table name regular expression can include: order, db, data, database, etc., and the node in the abstract syntax tree that records the logical table name can be identified by the table name regular expression.

[0138] The logical-physical mapping table records the mapping relationship between logical table names and physical table names. Based on the logical-physical mapping table, the logical table names in the nodes of the abstract syntax tree that record logical table names are modified to physical table names, resulting in the abstract syntax tree that serves as a rewrite list.

[0139] Furthermore, when the current data cluster expands by adding new databases, it needs to construct new mapping rules. This involves shuffling and re-splitting the data in all databases of the original data cluster to obtain new shards, which are then saved to all databases in the new data cluster. For example, if the original data cluster had 3 databases, the data was split using the ID%3 rule, and the resulting shards were saved in the 3 databases respectively. Once the data cluster expands to 4 databases, the ID%3 rule needs to be changed to ID%4. Therefore, all data in the original data cluster needs to be re-sharded according to the ID%4 rule, and the resulting shards are then saved in the 4 databases respectively.

[0140] This application addresses the issue of data cluster expansion by modifying the mapping between logical shard numbers and database and physical table names in the sharding rules. This modification establishes a mapping between certain logical shard numbers in the sharding rules and the newly added databases in the data cluster. This allows for the migration of business data corresponding to some logical shard numbers from the original databases to the new databases, significantly reducing the amount of data migration during expansion and greatly simplifying the process. For example, if the original data cluster includes databases 1, 2, and 3, adjusting the mapping rules in the sharding rules to migrate logical shards (database 1-table 3, database 2-table 1, database 3-table 2) to the newly added database 4, then the data cluster only needs to migrate databases 1-table 3, 2-table 1, and 3-table 2 to database 4, without migrating other data, thus greatly reducing the amount of data migration.

[0141] S204: Execute rewritten SQL statements in the target database to perform add, delete, modify, and / or query operations on logical shards in the target database.

[0142] In this step, by executing rewritten SQL statements in the target database, the logical shards are added, deleted, modified, and queried in the target database. This achieves the effect of using multiple databases to process massive amounts of business data, reduces the read and write pressure of each database, eliminates the risk of slow database operation, and avoids abnormal database downtime.

[0143] Furthermore, this application effectively alleviates the read / write pressure on a single database or table by splitting the data table into logical shards and storing these shards in multiple different databases, thereby enhancing the capacity of each unit to handle users and data. As the data-carrying capacity of each unit increases, the overall system unit size decreases accordingly.

[0144] Meanwhile, when a database reaches its storage limit and a new database needs to be added, the new database can be included in the management of the database sharding rules by modifying the logical shard number and its corresponding database name and physical table name, which greatly reduces the difficulty of expanding the data cluster.

[0145] Storing massive amounts of business data in a single database necessitates traversing multiple databases to retrieve the required data, significantly reducing query efficiency. In contrast, this application utilizes database sharding rules to determine the logical shards of the business data corresponding to the query request. These shards are then retrieved from each database and integrated to obtain the relevant business data, thus greatly improving query efficiency.

[0146] Optionally, executing rewrite SQL statements in the target database includes:

[0147] Check if the rewritten SQL statement is spelled correctly. If it is not, the target database will report a syntax error.

[0148] Checking the existence of accessed objects in the rewritten SQL statement. For example, if a column name is misspelled when writing a SELECT statement, the system will prompt an error. Syntax and semantic checks ensure that the rewritten SQL statement is error-free.

[0149] Check if the user has permission to access the data.

[0150] The shared pool is a memory pool whose primary function is to cache rewritten SQL statements and their execution plans. The target database determines whether to perform a soft parse or a hard parse by checking if the execution plan for the rewritten SQL statement exists in the shared pool. Within the shared pool, the target database first performs a hash operation on the rewritten SQL statement, then searches the library cache based on the hash value. If the execution plan exists, it is executed directly, entering the "executor" stage – this is soft parse. If neither the rewritten SQL statement nor the execution plan is found, the target database needs to create a parse tree for parsing, generate an execution plan, and enter the "optimizer" stage – this is hard parse. The optimizer is responsible for hard parses, deciding how to perform the parse, such as creating a parse tree and generating an execution plan. Once the parse tree and execution plan are available, the executor executes the parse tree according to the plan, achieving the effect of executing the rewritten SQL statement in the target database.

[0151] Specifically, rewriting SQL statements is performed in the target database, including:

[0152] Input the logical fragments into the target database;

[0153] The pre-built persistence layer framework is invoked to execute rewritten SQL statements in the target database to perform add, delete, modify, and / or query processing on logical shards in the target database.

[0154] For example, MyBatis is used as the persistence layer framework. MyBatis is an excellent persistence layer framework that supports custom SQL, stored procedures, and advanced mapping. MyBatis eliminates almost all JDBC code and the work of setting parameters and retrieving result sets. MyBatis can configure and map primitive types, interfaces, and Java POJOs (Plain Old Java Objects) to records in the database through simple XML or annotations. Therefore...

[0155] Example 2:

[0156] Please see Figure 3 This application provides a data sharding and database partitioning method, including:

[0157] S301: Obtain at least one database, and name each database to obtain at least one source name;

[0158] Create a data center server, and connect at least one database to the data center server to obtain a data cluster;

[0159] The source names of each database are compiled into a data list, which is then entered into the configuration file of the data center server.

[0160] In this step, each database is named according to a preset naming rule to obtain at least one source name. A data center server is created to connect to each database. The data center server is used to manage and control each database.

[0161] A data list is created based on the source name of each database to facilitate subsequent management and control of each database. This data list is then entered into a configuration file for the data center server to retrieve and access. Each database connected to the data center server is a sharded database; before sharding, it was a logical database from the application's perspective; after sharding, it becomes a collection of multiple physical databases.

[0162] S302: Obtain the configuration file, which contains at least one database name and the rule information for allocating logical shards to the databases corresponding to each database name;

[0163] The configuration file is parsed to obtain the database sharding and table partitioning rules, which are executable files containing the rule information.

[0164] In this step, a configuration file is a computer file that configures parameters and initial settings for some computer programs.

[0165] Preferably, the configuration file is an SDK (Software Development Kit), which is a collection of development tools used to build application software for specific software packages, software frameworks, hardware platforms, operating systems, etc. Running the configuration file installs the database sharding and table partitioning rules within it onto a server running data sharding and database partitioning methods, enabling these rules to run on that server and become executable files.

[0166] Optionally, parsing the configuration file to obtain the database sharding and table partitioning rules includes: instantiating a Loader, which is an inner class of ConfigFileApplicationListener. Let's look at the Loader object instantiation process: During the instantiation of the Loader object, two property file loaders are loaded again via SPI extension. One, YamlPropertySourceLoader, is closely related to the loading and parsing of the subsequent yml files, while the other, PropertiesPropertySourceLoader, is responsible for loading properties files. After creating the Loader instance, its load method is called. In the loadForFileExtension method, the yml (configuration file) is first loaded as a Resource file. Then, to prepare for the actual process, the load method of the previously created YamlPropertySourceLoader object is called. In the load method, the parsing and data encapsulation of the configuration file begins: the load method calls the load method of the OriginTrackedYmlLoader object, which in turn calls the getData interface of the OriginTrackingConstructor object to parse the yml (configuration file) and encapsulate it into an object. The Composer builder is actually used to generate nodes during the YAML parsing process. Its `getNode` method creates nodes via parser events. Typically, it encapsulates a set of data from the YAML into a `MappingNode`, which internally consists of a List of at least one `NodeTuple`. This `MappingNode`'s List contains the database sharding and table partitioning rules. `NodeTuple` has a similar structure to a Map, consisting of a pair of corresponding `keyNode` and `valueNode`. By converting the `MappingNode` into an executable file (`exe`), the database sharding and table partitioning rules are obtained as an executable file.

[0167] In this embodiment, py2exe, cx_freeze, or pyinstaller can be used to generate an executable file. An executable file is a file that can be loaded and executed by the operating system.

[0168] S303: Receive business data, perform sharding on the business data to obtain at least one logical shard, and determine the logical shard number of the logical shard based on the element string in the logical shard; wherein, business data is the data information used to input into the data cluster, the element string is used to describe the attribute characteristics of the sub-data in the logical shard, the element string consists of at least one business element or at least one business time, the business element is the metadata used to describe the sub-data, and the business time refers to the generation time of the sub-data.

[0169] This step is the same as S201 in Example 1, so it will not be described again here.

[0170] S304: Intercept the proxy object information corresponding to the logical shard, and extract the SQL statements in the proxy object information. The proxy object information is used to perform add, delete, modify, and query processing on the logical shard.

[0171] This step is the same as S202 in Example 1, so it will not be described again here.

[0172] S305: Based on the preset database sharding and table partitioning rules, obtain the database name and physical table name corresponding to the logical shard number, set the database corresponding to the database name in the data cluster as the target database, and modify the logical table name in the SQL statement to the physical table name to obtain the rewritten SQL statement. Here, the logical table name is the data table name indicated in the SQL statement, and the physical table name is the data table name defined by the logical table name in the target database.

[0173] This step is the same as S203 in Example 1, so it will not be described again here.

[0174] S306: Execute rewritten SQL statements in the target database to perform add, delete, modify, and / or query operations on logical shards in the target database.

[0175] This step is the same as S204 in Example 1, so it will not be described again here.

[0176] Example 3:

[0177] Please see Figure 4 This application provides a data sharding and database partitioning device 4, comprising:

[0178] The data sharding module 43 is used to receive business data, shard the business data to obtain at least one logical shard, and determine the logical shard number of the logical shard based on the element strings in the logical shard; wherein, business data is the data information used to input into the data cluster, and business elements are used to characterize the attribute features of the logical shard;

[0179] The input interception module 44 is used to intercept the proxy object information corresponding to the logical shard and extract the SQL statements in the proxy object information. The proxy object information is used to perform addition, deletion, modification and query processing on the logical shard.

[0180] The database sharding module 45 is used to obtain the database name and physical table name corresponding to the logical shard number according to the preset database sharding rules, set the database corresponding to the database name in the data cluster as the target database, and modify the logical table name in the SQL statement to the physical table name to obtain the rewritten SQL statement. Here, the logical table name is the data table name indicated in the SQL statement, and the physical table name is the data table name defined by the logical table name in the target database.

[0181] Statement execution module 46 is used to execute rewritten SQL statements in the target database to perform add, delete, modify, and / or query processing on logical shards in the target database.

[0182] Optionally, the data sharding and database partitioning device 4 includes:

[0183] Create module 41 to obtain at least one database, name each database to obtain at least one source name; create a data center server, connect each database to the data center server to obtain a data cluster; summarize the source names of each database to form a data list, and enter the data list into the configuration file of the data center server.

[0184] Configuration module 42 is used to obtain a configuration file, which contains at least one database name and rule information for allocating logical shards to the databases corresponding to each database name; the configuration file is parsed to obtain the database sharding and table partitioning rules, which are executable files containing rule information.

[0185] Example 4:

[0186] Figure 5 This is a block diagram of a computer device according to an exemplary embodiment. The computer device 5 may be a control circuit in an information management system 1. The control circuit is implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components in the information management system, and is used to execute a data fragmentation and database partitioning method.

[0187] Specifically, computer device 5 includes: a processor 52 and a memory 51 communicatively connected to the processor 52; the memory stores computer-executed instructions;

[0188] The processor executes computer execution instructions stored in memory 51 to implement the aforementioned data sharding and database partitioning method. The components of the data sharding and database partitioning device can be distributed across different computer devices. Computer device 5 can be a smartphone, tablet, laptop, desktop computer, rack server, blade server, tower server, or cabinet server (including standalone servers or server clusters composed of multiple application servers), etc. The computer devices in this embodiment include, but are not limited to, memory 51 and processor 52, which can be interconnected via a system bus. Figure 5 As shown. It should be noted that, Figure 5 Only computer devices with components are shown; however, it should be understood that it is not required to implement all of the shown components, and more or fewer components may be implemented instead. In this embodiment, memory 51 (i.e., readable storage medium) includes flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, memory 51 may be an internal storage unit of the computer device, such as the hard disk or memory of the computer device. In other embodiments, memory 51 may also be an external storage device of the computer device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., provided on the computer device. Of course, memory 51 may also include both internal storage units and external storage devices of the computer device. In this embodiment, memory 51 is typically used to store the operating system and various application software installed on the computer device, such as the program code of the data sharding and database partitioning device of Embodiment 3. In addition, the memory 51 can also be used to temporarily store various types of data that have been output or will be output. In some embodiments, the processor 52 can be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip. The processor 52 is typically used to control the overall operation of the computer device. In this embodiment, the processor 52 is used to run program code stored in the memory 51 or process data, for example, to run a data fragmentation and database partitioning device to implement the data fragmentation and database partitioning method of the above embodiments.

[0189] The integrated modules implemented as software functional modules described above can be stored in a computer-readable storage medium. These software functional modules, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods of the various embodiments of this application. It should be understood that the processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in this application can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules in the processor. The memory may include high-speed RAM, and may also include non-volatile memory (NVM), such as at least one disk storage device, and may also be a USB flash drive, external hard drive, read-only memory, disk, or optical disk, etc.

[0190] To achieve the above objectives, this application also provides a computer-readable storage medium, such as flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, server, App application store, etc., which stores computer-executable instructions, and the program implements the corresponding function when executed by processor 52. The computer-readable storage medium of this embodiment is used to store computer-executable instructions for implementing the data fragmentation and database partitioning method, and when executed by processor 52, it implements the data fragmentation and database partitioning method of the above embodiment.

[0191] The aforementioned storage medium can be implemented from any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The storage medium can be any available medium accessible to general-purpose or special-purpose computers.

[0192] An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Alternatively, the storage medium can be an integral part of the processor. The processor and storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and storage medium can exist as discrete components within a computer device or host device.

[0193] This application provides a computer program product, including a computer program that, when executed by a processor, implements the above-described data fragmentation and database partitioning method.

[0194] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0195] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0196] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A data sharding and database partitioning method, characterized in that, include: Receive business data, perform fragmentation on the business data to obtain at least one logical fragment, extract at least one business element from the logical fragment, and concatenate at least one business element to obtain an element string; Alternatively, extract at least one business time from the logical segments and concatenate at least one of the business times to obtain an element string; The element string is processed according to the preset sharding and splitting rules to obtain the library shard number and the table shard number. The logical shard number is obtained by concatenating the library shard number and the table shard number. or The element string is processed according to a preset hash algorithm to obtain a hash value. The hash value is then processed according to preset database and table operation rules to obtain a database shard number and a table shard number. These two numbers are then concatenated to obtain the logical shard number of the logical shard. The business data refers to the data information input into the data cluster. The element string describes the attribute characteristics of the sub-data in the logical shard. The element string consists of at least one business element or at least one business time. The business element is metadata describing the sub-data, and the business time refers to the generation time of the sub-data. Intercept the proxy object information corresponding to the logical shard, and extract the SQL statement in the proxy object information, wherein the proxy object information is used to perform add processing, and / or delete processing, and / or modify processing, and / or query processing on the logical shard; Extract the database shard number from the logical shard number, and obtain the database name that matches the database shard number from the preset database sharding and table partitioning rules; the database sharding and table partitioning rules include the mapping relationship between the logical shard number and the database name and physical table name; Extract the table shard number from the logical shard number, obtain the physical table name corresponding to the table shard number from at least one physical table name corresponding to the database name in the database sharding rules, set the database corresponding to the database name in the data cluster as the target database, and modify the logical table name in the SQL statement to the physical table name to obtain the rewritten SQL statement. The target database is used to store the logical shard; the logical table name is the data table name indicated in the SQL statement, and the physical table name is the data table name defined by the logical table name in the target database. The rewritten SQL statement is executed in the target database to perform add, delete, modify, and / or query processing on the logical shard in the target database.

2. The data shard library distribution method of claim 1, wherein, The business data is sharded to obtain at least one logical shard, including: Identify the element type of each business element in the business data, and aggregate at least one business element with the same element type into an element set, wherein the element type represents the category of the business element; The column data corresponding to the business elements in each of the element sets is extracted from the business data to obtain the logical fragments corresponding to each of the element sets, wherein the column data is the sub-data corresponding to each business element in the business data.

3. The data sharding and library distributing method of claim 1, wherein, The business data is sharded to obtain at least one logical shard, including: Based on the business time of the row data, at least one row of data in the business data is sorted in ascending or descending order to obtain a business sequence, wherein the row data is sub-data generated at different times in the business data; The time span information of the business data is obtained based on the business time at the beginning of the business sequence and the business time at the end of the business sequence. The time span information is divided according to the preset number of segments to obtain time periods corresponding to the number of segments; From the business time of acquiring each row of data, and summing the business times belonging to the same time period, at least one time set is obtained; Extract the row data corresponding to the business time in each of the time sets from the business data to obtain the logical slices corresponding to each of the time sets.

4. The data sharding and library distributing method of claim 1, wherein, Intercepting proxy object information corresponding to the logical shard includes: If a mapping interface is detected to be called, the proxy object information received in the mapping interface is intercepted and the proxy fragment number in the proxy object information is extracted. The mapping interface is a computer interface used to process business data corresponding to the proxy object information based on the proxy object information. The proxy object information whose proxy shard number matches the logical shard number of the logical shard is set as the proxy object information corresponding to the logical shard.

5. The data sharding and library distributing method of claim 1, wherein, Setting the database in the data cluster corresponding to the database name as the target database includes: Obtain a data list from the data cluster, wherein the data list contains the source name of at least one database; Iterate through the data list and set the database whose source name matches the database name as the target database; Establish a communication connection with the target database, the communication connection being a communication line used to access the target database.

6. The data sharding and library distributing method of claim 1, wherein, The rewritten SQL statement is obtained by modifying the logical table name in the SQL statement to the physical table name, including: The SQL statement is parsed to obtain an abstract syntax tree with at least one instruction vocabulary. The parsing refers to performing syntactic analysis and / or semantic analysis on the SQL statement to obtain statement logic, and decomposing the SQL statement according to the statement logic to obtain computer rules with at least one instruction having a specific meaning. Identify the logical table name in the abstract syntax tree, modify the logical table name to the physical table name, and generate a rewritten SQL statement based on the abstract syntax tree with the physical table name.

7. The data shard library distribution method of claim 6, wherein, Identify the logical table names in the abstract syntax tree, modify the logical table names to the physical table names, and generate rewritten SQL statements based on the abstract syntax tree containing the physical table names, including: Each node in the abstract syntax tree is identified by a pre-defined table name regular expression, and the node containing the logical table name is obtained. The table name regular expression contains keywords that represent the logical table name. The logical table name is modified to the physical table name to obtain an abstract syntax tree with the physical table name. Based on the syntax logic of the abstract syntax tree, the abstract syntax tree with the physical table name is converted into a rewritten SQL statement.

8. The data fragmentation and database partitioning method according to any one of claims 1-7, characterized in that, Before receiving business data, the method further includes: Obtain at least one database, and name each database to obtain at least one source name; Create a data center server, and connect at least one of the databases to the data center server to obtain a data cluster; The source names of each database are compiled to form a data list, which is then entered into the configuration file of the data center server.

9. The data shard library distribution method of claim 8, wherein, After entering the data list into the configuration file of the data center server, the method further includes: Obtain the configuration file, which contains at least one of the database names and rule information for allocating logical shards to the databases corresponding to each of the database names; The configuration file is parsed to obtain the database sharding and table partitioning rules, which are executable files containing the rule information.

10. A data sharding and distributing device, characterized in that, include: The data sharding module is used to receive business data, shard the business data to obtain at least one logical shard, extract at least one business element from the logical shard, and concatenate at least one business element to obtain an element string. Alternatively, extract at least one business time from the logical segments and concatenate at least one of the business times to obtain an element string; The element string is processed according to the preset sharding and splitting rules to obtain the library shard number and the table shard number. The logical shard number is obtained by concatenating the library shard number and the table shard number. or The element string is processed according to a preset hash algorithm to obtain a hash value. The hash value is then processed according to preset database and table operation rules to obtain a database shard number and a table shard number. The database shard number and the table shard number are then concatenated to obtain the logical shard number of the logical shard. The business data is the data information used to input into the data cluster, and the business elements are used to characterize the attribute features of the logical shard. The input interception module is used to intercept the proxy object information corresponding to the logical shard and extract the SQL statements in the proxy object information, wherein the proxy object information is used to perform addition processing, and / or deletion processing, and / or modification processing, and / or query processing on the logical shard; The database sharding module is used to extract the database shard number from the logical shard number and obtain the database name that matches the database shard number from the preset database sharding rules; the database sharding rules include the mapping relationship between the logical shard number and the database name and physical table name; Extract the table shard number from the logical shard number, obtain the physical table name corresponding to the table shard number from at least one physical table name corresponding to the database name in the database sharding rules, set the database corresponding to the database name in the data cluster as the target database, and modify the logical table name in the SQL statement to the physical table name to obtain the rewritten SQL statement. The target database is used to store the logical shard; the logical table name is the data table name indicated in the SQL statement, and the physical table name is the data table name defined by the logical table name in the target database. The statement execution module is used to execute the rewritten SQL statement in the target database to perform add, delete, modify, and / or query processing on the logical shard in the target database.

11. A computer device, comprising: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1 to 9.

13. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, is used to implement the method as described in any one of claims 1 to 9.

Images