Database splitting method and device, equipment and storage medium

By using the transaction target number in the bank transaction system to determine the target database and perform data migration, data conflicts and inconsistencies during database splitting are solved, and database splitting and data consistency are achieved without suspending transactions.

CN119088882APending Publication Date: 2024-12-06CHINA MERCHANTS BANK

Patent Information

Application Number
CN202411192401.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-28
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

In a bank transaction system, when the database is split, the performance and business status indicators exceed the preset range, resulting in customer transactions being unable to be processed correctly, and data conflicts or inconsistencies may occur, and transactions need to be suspended.

Method used

During the database splitting process, the target database is determined using the transaction target number, and the transaction request and data are directly written to the corresponding database, and the historical data is transplanted after completing the traffic switching, reducing the dependence between traffic switching and data migration, ensuring data Integrity and consistency.

Benefits of technology

It realizes database splitting without affecting customer transactions, improves splitting efficiency and data consistency, and avoids data conflicts and inconsistencies.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119088882A_ABST
    Figure CN119088882A_ABST
Patent Text Reader

Abstract

The invention discloses a database splitting method, device and equipment and a storage medium, and relates to the technical field of database splitting, and the method comprises the steps: when a transaction request initiated by a client is received in a database splitting process, determining a target sub-database based on the number of a transaction object of the transaction request; distributing the transaction request to the target sub-library, and storing data generated by the transaction request to the target sub-library; and after transaction request distribution is completed, historical data of the old database are determined, and the historical data are transplanted to the corresponding sub-databases according to the transaction object numbers of the historical data. According to the method and the device, the transaction of the customer is not suspended in the database splitting process.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of database splitting, and in particular to a database splitting method, device, equipment and storage medium. Background Art

[0002] In banking transaction systems, in order to cope with the continuous growth of business volume and the improvement of system security requirements, the strategy of splitting a single database system into multiple sub-databases is usually adopted.

[0003] In the Chinese invention patent with publication number CN111930741A, when splitting a database, when the indicator data showing the performance of the database and the status of executing the business exceeds the preset data range, the range interval of the area number of the data table is used as the database splitting rule, and the data table with the area number within a certain interval corresponds to a sub-library. In this process, if customer transactions are allowed, the new data generated by customer transactions may not be properly processed during the splitting process, resulting in data conflicts or inconsistencies. Therefore, customer transactions need to be suspended. Summary of the invention

[0004] The main purpose of this application is to provide a database splitting method, device, equipment and storage medium, aiming to solve how to split the database without affecting customer transactions.

[0005] To achieve the above purpose, the present application proposes a database splitting method, which comprises:

[0006] When receiving a transaction request initiated by a customer during the database splitting process, determining a target sub-database based on the transaction subject number of the transaction request;

[0007] Distribute the transaction request to the target sub-database, and store the data generated by the transaction request in the target sub-database;

[0008] After completing the transaction request distribution, the historical data of the old database is determined, and the historical data is transplanted to the corresponding sub-databases according to the transaction subject number of the historical data.

[0009] In one embodiment, the step of determining the target sub-database based on the transaction subject number of the transaction request includes:

[0010] Parsing the transaction request to obtain information of the transaction object, and generating a number of the transaction object according to the information of the transaction object;

[0011] Reading a preset routing strategy, and determining the target sub-database according to the transaction subject number and the routing strategy;

[0012] Distribute the transaction request to the target sub-database.

[0013] In one embodiment, the step of determining the target sub-database based on the transaction subject number of the transaction request further includes:

[0014] When receiving a transfer target request initiated by the transferor, parsing the transfer target request to obtain information about the transfer target;

[0015] Generating the transaction subject number based on the information of the transfer subject;

[0016] Read the preset routing strategy, and determine the target sub-database according to the transaction subject number and the routing strategy;

[0017] Distribute the transfer target request to the target sub-library.

[0018] In one embodiment, after the step of distributing the transfer target request to the target sub-database, the method further includes:

[0019] When receiving a purchase request initiated by a purchaser, parsing the purchase request to obtain a transaction target number;

[0020] Reading the preset routing strategy, and determining the target sub-database according to the transaction subject number and the routing strategy;

[0021] Distribute the purchase request to the target sub-database.

[0022] In one embodiment, before the step of determining the target sub-database based on the transaction subject number of the transaction request, the method further includes:

[0023] Configure the sub-library rules and establish a mapping relationship between each sub-library and the transaction target number.

[0024] In one embodiment, the step of determining the historical data of the old database and transplanting the historical data to the corresponding sub-databases according to the preset transaction subject numbers of the historical data includes:

[0025] Acquire the transaction target information of the historical data, and generate the transaction target number of the historical data according to the transaction target information of the historical data;

[0026] Read the sub-library rule, and determine the sub-library that has a mapping relationship with the transaction subject number of the historical data according to the sub-library rule and the transaction subject number of the historical data;

[0027] The historical data is transplanted to the sub-database that has a mapping relationship with the transaction subject number of the historical data.

[0028] In one embodiment, the method further includes:

[0029] Monitor the operation status of each of the sub-libraries, and when an abnormality occurs in a first sub-library, obtain information about the first sub-library and a transaction target number that is mapped to the first sub-library;

[0030] Determine a second sub-database, and establish a new mapping relationship between the second sub-database and the transaction subject number that has a mapping relationship with the first sub-database;

[0031] Update the routing strategy based on the new mapping relationship;

[0032] When receiving a transaction request from a customer, the transaction request is forwarded to the second data sub-database according to the updated routing strategy.

[0033] In addition, to achieve the above purpose, the present application also proposes a database splitting device, the database splitting device comprising:

[0034] A target sub-database determination module, configured to determine a target sub-database based on a transaction subject number of a transaction request when receiving a transaction request initiated by a customer during the database splitting process;

[0035] A transaction request distribution module, used to distribute the transaction request to the target sub-database, and store the data generated by the transaction request in the target sub-database;

[0036] The historical data migration module is used to determine the historical data of the old database after completing the transaction request distribution, and migrate the historical data to the corresponding sub-databases according to the transaction target number of the historical data.

[0037] In addition, to achieve the above-mentioned purpose, the present application also proposes a database splitting device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the database splitting method described above.

[0038] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the database splitting method described above are implemented.

[0039] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the database splitting method described above are implemented.

[0040] The present application provides a database splitting method. First, when a transaction request initiated by a customer is received in the database splitting process, a target sub-library is determined based on the transaction subject number of the transaction request. By using the transaction subject number to perform the target sub-library, it is possible to process and store related transaction data of the same subject in the same sub-library, thereby reducing the dependency between transactions and customers. Secondly, the transaction request is distributed to the target sub-library, and the data generated by the transaction request is stored in the target sub-library. The newly generated data of the transaction request is directly written into the corresponding data sub-library, and the old database does not receive new transaction data. Therefore, real-time data synchronization is not required, which reduces the data that needs to be transplanted and improves the efficiency of database splitting. Finally, after completing the transaction request distribution, the historical data of the old database is determined, and the historical data is transplanted to the corresponding sub-libraries according to the transaction subject number of the historical data, so as to ensure the integrity and consistency of the data.

[0041] In summary, this application first switches the traffic flow, switches the transaction request initiated by the customer to the corresponding sub-database through the transaction subject number, and directly writes the newly generated data of the transaction request to the corresponding sub-database, and the old data base no longer receives new data; after completing the traffic switch, the historical data in the old data base is transplanted to the corresponding data sub-database, which reduces the dependence of traffic switching and data transplantation during the database splitting process and improves the efficiency of database splitting. Through the above steps, the database splitting can be realized without suspending the customer's transaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0043] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0044] Figure 1 A schematic diagram of a process flow provided for the first embodiment of the database splitting method of the present application;

[0045] Figure 2 A schematic diagram of request distribution before traffic switching provided in Example 1 of the database splitting method of the present application;

[0046] Figure 3 A schematic diagram of request distribution during the traffic switching process provided in Example 1 of the database splitting method of the present application;

[0047] Figure 4A first detailed flow chart of the method for splitting a database according to the present application is provided in Example 1;

[0048] Figure 5 A second detailed flow chart provided for the first embodiment of the database splitting method of the present application;

[0049] Figure 6 A third detailed flow chart provided for the first embodiment of the database splitting method of the present application;

[0050] Figure 7 A schematic diagram of the process flow provided in Example 3 of the database splitting method of this application;

[0051] Figure 8 This is a schematic diagram of the module structure of the database splitting device according to an embodiment of the present application;

[0052] Fig. 9 This is a schematic diagram of the device structure of the hardware operating environment involved in the database splitting method in the embodiment of the present application.

[0053] Description of Figure Numbers:

[0054] 10. Target sub-database determination module; 20. Transaction request distribution module; 30. Historical data migration module; 1001. Processing device; 1002. Read-only memory; 1003. Storage device; 1004. Memory; 1005. Bus; 1006. I / O interface; 1007. Input device; 1008. Output device; 1009. Communication device.

[0055] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0056] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0057] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0058] The main solution of the embodiment of the present application is: to obtain the transaction request initiated by the customer, and distribute the transaction request to the corresponding preset data sub-library according to the preset routing strategy; to store the data generated by the transaction request in the preset data sub-library that receives the transaction request; to determine the historical data of the old database, and to transplant the historical data to the corresponding preset data sub-libraries according to the preset sub-library rules.

[0059] In the prior art, when splitting a database, when the indicator data showing the performance of the database and the status of business execution exceeds the preset data range, the database is split using the range interval of the area number of the data table or the area number field value of the data table as the database splitting rule. In this process, if customer transactions are allowed, the new data generated by the customer transactions may not be properly processed during the splitting process, resulting in data conflicts or inconsistencies. Therefore, customer transactions need to be suspended.

[0060] This application first switches the traffic flow, switches the transaction request initiated by the customer to the corresponding sub-database through the transaction target number, and directly writes the newly generated data of the transaction request to the corresponding sub-database, and the old data base no longer receives new data; after completing the traffic switch, the historical data in the old data base is transplanted to the corresponding data sub-database, which reduces the dependence of traffic switching and data transplantation during the database splitting process and improves the efficiency of database splitting. Through the above steps, the database splitting can be realized without suspending the customer's transaction.

[0061] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, a program process abnormality monitoring device, etc. The following takes the database splitting device as an example to illustrate this embodiment and the following embodiments.

[0062] Based on this, the embodiment of the present application provides a database splitting method, referring to Figure 1 , Figure 1 This is a flowchart of the first embodiment of the database splitting method of the present application.

[0063] In this embodiment, the database splitting method includes steps S10 to S30:

[0064] Step S10, when receiving a transaction request initiated by a customer during the database splitting process, determining a target sub-database based on the transaction subject number of the transaction request;

[0065] It should be noted that database splitting refers to the process of dividing a large database into multiple smaller databases according to certain rules or strategies. Each database instance is called a sub-database. The transaction target number refers to a string of codes or numbers used to uniquely identify and distinguish different transaction objects (such as stocks, bonds, futures, options, commodities, etc.) in the financial market or trading system. These numbers are usually formulated by exchanges, regulatory agencies or market participants according to certain rules and formats to facilitate the smooth progress of transactions, clearing, settlement and information disclosure. A routing gateway is a network node that acts as a bridge between two or more different networks, allowing devices in these networks to communicate. In the TCP / IP protocol, a gateway is essentially an IP address that leads from one network to another network. The IP address of a device with routing function (such as a router) is the gateway address.

[0066] In this embodiment, during the process of database sub-library, when a customer initiates a new transaction request through a front-end application or a client, the transaction request is sent to the routing gateway. After receiving the transaction request, the routing gateway determines which sub-library the transaction request should be forwarded to based on the transaction subject number corresponding to the transaction request. By using the transaction subject number as the basis for forwarding the transaction request, it is possible to process and store all transaction data related to the same subject in the same sub-library, thereby reducing the dependency between transactions and customers; at the same time, when querying data, it is possible to reduce the need for cross-library queries, improve the locality of data access, and speed up the query speed.

[0067] Please refer to Figure 4 In a feasible implementation manner, step S10 may include steps A11 to A13:

[0068] Step A11, parsing the transaction request to obtain transaction target information, and generating a transaction target number according to the transaction target information;

[0069] Step A12, reading a preset routing strategy, and determining the target sub-database according to the transaction subject number and the routing strategy;

[0070] Step A13: distribute the transaction request to the target sub-database.

[0071] It should be noted that the transaction subject refers to the transaction object discussed, agreed upon and determined by the two parties during the transaction process, which can be stocks, bonds, currencies, futures and options, or retail goods. Routing strategy is a technology that modifies routing information in order to change the path of network traffic, mainly by changing routing attributes (including reachability).

[0072] In this implementation, first, after the routing gateway receives the transaction request, the transaction request sent by the customer is parsed to extract the information of the transaction tag used for routing, and the transaction tag number is generated according to the information of the transaction tag and the numbering rule. The transaction tag number can uniquely identify and distinguish different transaction objects; secondly, the currently effective preset routing policy is queried, and the routing policy contains matching information about which sub-database to forward the transaction request corresponding to the transaction tag number to. For example, a certain interval segment of the transaction tag number (such as the last few digits of the number) is used to determine the target sub-database with a mapping relationship with the transaction tag number; finally, the forwarding action is executed according to the routing policy to distribute the transaction request to the target sub-database. The routing policy may be stored in a configuration center, a database, or a memory, and can be dynamically adjusted according to the load, health status, or business needs of the data sub-database.

[0073] In this implementation, the transaction request initiated by the customer is distributed to the corresponding data sub-database through the transaction target number. Compared with the request distribution based on customer information, the transaction target number is used as the basis for traffic switching. The transaction target number usually has a finer-grained distinction and can more accurately control the distribution of traffic, especially in high-concurrency scenarios, which helps to achieve a more balanced load distribution. In addition, using customer information as the basis for traffic switching may cause the amount of data of some popular customer information to be too large, forming data hotspots, while the transaction target number can usually distribute data more evenly and avoid data skew problems. Since the transaction target number is closely related to specific business operations, using it as the basis for traffic switching can more easily ensure data consistency and integrity.

[0074] Please refer to Figure 5 In another feasible implementation, step S10 may include steps B11 to B14:

[0075] Step B11, upon receiving a transfer target request initiated by the transferor, parsing the transfer target request to obtain information on the transfer target;

[0076] Step B12, generating the transaction subject number based on the information of the transfer subject;

[0077] Step B13, reading the preset routing strategy, and determining the target sub-database according to the transaction subject number and the routing strategy;

[0078] Step B14, distribute the transfer subject request to the target sub-library.

[0079] In this embodiment, after the routing gateway obtains the transfer target request initiated by the transferor, it parses the transfer target request and generates a transaction target number based on the parsing result. The transaction target number is used to uniquely identify a transaction. Secondly, according to the mapping relationship between the transaction target number contained in the routing strategy and the preset data sub-library, it is determined to which sub-library the transfer target request is forwarded, and the transfer target request is distributed. Through the above steps, the transaction request newly initiated by the transferor during the database splitting process is distributed to the data sub-library, and the newly generated transaction request and related data are no longer recorded in the old data library, which reduces the amount of data that needs to be migrated for database splitting and improves the efficiency of database splitting.

[0080] Please refer to Figure 5 In another feasible implementation manner, step B14 may further include steps B15 to B17:

[0081] Step B15, upon receiving a purchase request initiated by the purchaser, parsing the purchase request to obtain the transaction target number;

[0082] Step B16, reading the preset routing strategy, and determining the target sub-database according to the transaction subject number and the routing strategy;

[0083] Step B17, distribute the purchase target request to the target sub-database.

[0084] In this embodiment, after the routing gateway obtains the purchase target request initiated by the purchaser, it parses the purchase target request and generates a transaction target number based on the analysis result; secondly, it determines to which sub-library the purchase target request is forwarded based on the transaction target number contained in the routing strategy and the preset routing strategy, so as to distribute the purchase target request.

[0085] In this implementation, according to the transaction subject number, the transfer subject request initiated by the transferor is first switched to the corresponding data sub-library, and then the subsequent purchase request initiated by the buyer is switched to the same data sub-library, and the transactions between the transferor and the buyer are stored in one data sub-library, ensuring that the relevant transaction data of the same subject are processed and stored in the same data sub-library, and all data of the same transaction are completely recorded in the same data sub-library.

[0086] The present embodiment provides a database splitting method, which switches the transaction request initiated by the customer to the corresponding data sub-database first, and directly writes the newly generated data of the transaction request into the corresponding data sub-database, and the old data base no longer receives new data; after completing the traffic switching, the historical data in the old data base is transplanted to the corresponding data sub-database, thereby reducing the dependence of traffic switching and data transplantation during the database splitting process, and realizing database splitting without suspending customer transactions.

[0087] Step S20, distributing the transaction request to the target sub-database, and storing the data generated by the transaction request in the target sub-database;

[0088] In this embodiment, during the process of database sub-library, when the customer initiates a new transaction request through the front-end application or client, the transaction request is sent to the routing gateway. After receiving the transaction request, the routing gateway first parses it to extract the transaction subject for routing, etc., to determine which data sub-library the transaction request should be forwarded to; secondly, query the currently effective routing strategy, which may be stored in the configuration center, database or memory. The routing strategy can be dynamically adjusted according to the load, health status or business needs of the data sub-library; finally, based on the information obtained from the analysis of the transaction request and the routing strategy, determine the data sub-library to which the transaction request should be forwarded, and the routing gateway forwards the transaction request to the corresponding data sub-library. In the process of database splitting, traffic switching is first performed to distribute the new transaction request initiated by the user to the corresponding data sub-library. During the traffic switching, there is no need to suspend the customer's transaction, and the continuity of the customer's transaction is not affected.

[0089] In this embodiment, the routing policy may also include a dynamic allocation mechanism, for example, using a proxy server for dynamic allocation, such as: selecting Nginx or HAProxy as a load balancer, and forwarding requests to multiple MySQL database instances according to the configured policy. For example, by configuring the upstream block of Nginx to specify the addresses of multiple database instances, and then using a load balancing algorithm (such as polling, least number of connections, etc.) to allocate requests. When the load of a sub-library is too high, the routing policy can automatically distribute new requests to other sub-libraries with lower loads to ensure the overall performance of the system.

[0090] Step S30, after completing the transaction request distribution, determine the historical data of the old database, and transplant the historical data to the corresponding sub-databases according to the transaction subject number of the historical data.

[0091] It should be noted that the old database usually refers to the database instance established in the early stage of the database system and used to store data. It carries all the historical data of the system since its establishment. In the process of database splitting, the historical data of the old database mainly refers to the business-related data that has been accumulated and stored in the database over the past period of time and is no longer frequently changed or accessed. The historical data should maintain integrity and accuracy to ensure that it can accurately reflect the past business conditions when needed. Therefore, in the process of database splitting, the historical data should be properly processed and migrated to avoid data loss or damage.

[0092] Please refer to Figure 6 In a feasible implementation manner, step S30 may include steps S31 to S33:

[0093] Step S31, obtaining the transaction target information of the historical data, and generating the transaction target number of the historical data according to the transaction target information of the historical data;

[0094] In this implementation, first, the transaction target information of each transaction record is extracted from the historical data. The transaction target information may refer to a specific financial product, commodity or service, which uniquely identifies the object of the transaction. Secondly, based on the transaction target information, the transaction target number is generated using the transaction target information and the coding rules. The transaction target number can uniquely represent each transaction target.

[0095] Step S32, reading the sub-library rule, and determining the sub-library that has a mapping relationship with the transaction subject number of the historical data according to the sub-library rule and the transaction subject number of the historical data;

[0096] Step S33, transplanting the historical data to the sub-library that has a mapping relationship with the transaction subject number of the historical data.

[0097] In this implementation, first, the library sub-library rules are read, which define how to map different transaction target numbers to different sub-libraries. Optionally, the library sub-library rules define the sub-libraries corresponding to a certain interval of the transaction target number (such as the last few digits of the number). Secondly, according to the library sub-library rules and the transaction target number, the sub-library corresponding to each transaction target number is calculated or searched.

[0098] Optionally, the historical data of the old database can be exported using ETL (Extract, Transform, Load) tools. After extracting the historical data, the data is cleaned, including removing duplicate data, processing missing values, correcting erroneous data, etc., to ensure the accuracy and completeness of the data. After completing the data migration, verification and testing are performed to ensure that the data is correctly distributed to each data sub-library. Exemplarily, first, data consistency verification is performed to calculate the number of data rows in a specific table or the entire database in each data sub-library, and compare it with the corresponding data in the old database before migration to ensure that the number of data rows is consistent. When performing data content verification, some data is randomly selected for detailed comparison to check whether the specific content of the data is consistent, especially the data of key fields and complex data types. Secondly, the sub-library rules are verified by writing query scripts or using database management tools to check the distribution of data and verify whether the data is correctly distributed to each data sub-library.

[0099] Exemplarily, there are 4 sub-databases, namely DB1, DB2, DB3 and DB4. The transaction target number is a 10-digit string. The last 3 digits of the number are used to determine in which sub-database the historical data should be stored. The preset sub-database rule can be: if the last 3 digits of the transaction target number are between 000 and 299 (including 000 and 299), the data is stored in DB1; if the last 3 digits of the transaction target number are between 300 and 599 (including 300 and 599), the data is stored in DB2; if the last 3 digits of the transaction target number are between 600 and 899 (including 600 and 899), the data is stored in DB3; if the last 3 digits of the transaction target number are between 900 and 999 (including 900 and 999), the data is stored in DB4.

[0100] Exemplarily, there are 4 sub-databases, namely DB1, DB2, DB3 and DB4. The preset sub-database rules can also be: the transaction target number is a string of letters and numbers, and the sub-database is determined according to the hash value of the transaction target number. In the hash function, the transaction target number is used as input and an integer between 1 and 4 is output. If the output is 1, the corresponding historical data is transplanted to DB1; if the output is 2, the corresponding historical data is transplanted to DB2; if the output is 3, the corresponding historical data is transplanted to DB3; if the output is 4, the corresponding historical data is transplanted to DB4.

[0101] In this implementation, after the distribution of customer requests is completed, the historical data in the old database is migrated, and based on the transaction subject number corresponding to the historical data as the basis for database splitting, all the historical data is migrated to the corresponding data sub-database to complete the database splitting. During the process of historical data migration, there are no new customer requests or new data in the old database, which can improve the efficiency of database splitting and data consistency.

[0102] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated later. On this basis, before step S10, the database splitting method further includes the following steps:

[0103] Configure the sub-library rules and establish a mapping relationship between each sub-library and the transaction target number.

[0104] In this embodiment, the transaction target number corresponding to each piece of historical data is used as a database partitioning rule to split the database, and a mapping relationship is established between the transaction target number and the data sub-library. The mapping relationship can be a many-to-one mapping relationship between multiple transaction target numbers and different data sub-libraries. Exemplarily, when configuring the sub-library rules, the mapping relationship between the target number and the database is defined. First, a unique target number is defined for each sub-library. This number can be a number, a letter, or a combination thereof, which is used to uniquely identify the target sub-library. Secondly, a mapping relationship is established between the transaction target number and the corresponding sub-library, and the mapping relationship is stored in a configuration file or a database table. The mapping relationship contains the correspondence between the transaction target number and the database connection information.

[0105] In this embodiment, using the transaction subject number as the database partitioning rule instead of using the customer information as the database partitioning rule has the following benefits and advantages: more even data distribution: using the transaction subject number as the database partitioning key can avoid the problem of excessive load on some databases due to uneven distribution of customer information. The transaction subject number is usually more closely related to business operations, which helps to achieve uniform distribution of data; high query efficiency: when the query involves the transaction subject, the specific database can be directly located according to the transaction subject number without the need to query across multiple databases, which can improve query efficiency; easy to expand: using the transaction subject number as the database partitioning rule can make it easier to horizontally expand the database when the business develops and the transaction volume increases, for example, by adding more databases to store new transaction data; reducing data migration: avoiding data hot spot problems: using customer information as the database partitioning rule may cause the amount of data for some popular customers to be too large, forming data hot spots and affecting database performance, while the transaction subject number can usually distribute data more evenly and avoid data hot spot problems; easy to achieve data consistency and integrity: since the transaction subject number is closely related to specific business operations, using it as a database partitioning rule can make it easier to ensure data consistency and integrity.

[0106] Based on the first embodiment of the present application, in the third embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. Figure 7 The database splitting method further includes steps C10 to C40:

[0107] Step C10, monitoring the operating status of each sub-library, and when a first sub-library is abnormal, obtaining information of the first sub-library and a transaction subject number that is mapped to the first sub-library;

[0108] It should be noted that the first database refers to a database whose operating status is abnormal.

[0109] In this embodiment, the operating status of each data sub-database is monitored based on a monitoring tool. If there is an abnormality in the operating status of a data sub-database, the first data sub-database where the abnormality occurs is quickly located, and the transaction target number that has a mapping relationship with the first data sub-database in the routing strategy is determined.

[0110] Optionally, a monitoring tool is selected to monitor the operating status of each data sub-database, and the monitoring tool may be OEM (Oracle Enterprise Manager), SSMS (SQL Server Management Studio), Prometheus, Zabbix, etc. The monitoring tool is used to collect data and monitor indicators, and the data and monitor indicators may include: number of connections, response time, CPU usage, memory usage, disk I / O, lock waiting time, and error log, wherein the number of connections indicates the number of currently active connections; the response time indicates the response time of a query or operation; the CPU usage indicates the usage of the database server CPU; the memory usage indicates the usage of the database server memory; the disk I / O indicates the frequency and performance of disk read and write operations; the lock waiting time indicates the time a transaction or query waits for a lock; and the error log indicates the error information in the database error log.

[0111] Step C20, determining a second sub-database, and establishing a new mapping relationship between the second sub-database and the transaction subject number that has a mapping relationship with the first sub-database;

[0112] Step C30, updating the routing strategy based on the new mapping relationship;

[0113] Step C40: when receiving a transaction request from a customer, forwarding the transaction request to the second data sub-database according to the updated routing strategy.

[0114] In this embodiment, a healthy data sub-database is selected as the second data sub-database, a new mapping relationship is formed between the transaction target number that has a mapping relationship with the first data sub-database and the second data sub-database, and the routing strategy is updated based on the new mapping relationship formed between the transaction target number and the second data sub-database. After the first sub-database is abnormal, the routing gateway forwards the transaction request of the transfer target that originally belonged to the first data sub-database to the second data sub-database according to the updated routing strategy, so that the second data sub-database takes over the customer request originally from the first data sub-database, so that the abnormality of the first data sub-database does not affect the normal transaction application of the customer, and the availability of the distributed system is improved.

[0115] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the database splitting method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0116] This application also provides a database splitting device, please refer to Figure 8 , the database splitting device comprises:

[0117] The target sub-database determination module 10 is used to determine the target sub-database based on the transaction subject number of the transaction request when receiving a transaction request initiated by a customer during the database splitting process;

[0118] A transaction request distribution module 20, configured to distribute the transaction request to the target sub-database, and store the data generated by the transaction request in the target sub-database;

[0119] The historical data migration module 30 is used to determine the historical data of the old database after completing the transaction request distribution, and migrate the historical data to the corresponding sub-databases according to the transaction target number of the historical data.

[0120] The database splitting device provided by the present application adopts the database splitting method in the above embodiment, which can solve the problem of how to split the database without affecting customer transactions. Compared with the prior art, the beneficial effects of the database splitting device provided by the present application are the same as the beneficial effects of the database splitting method provided by the above embodiment, and other technical features in the database splitting device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0121] The present application provides a database splitting device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the database splitting method in the above-mentioned embodiment 1.

[0122] Reference below Fig. 9 , which shows a schematic diagram of the structure of a database splitting device suitable for implementing the embodiment of the present application. The database splitting device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Fig. 9 The database splitting device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0123] like Fig. 9As shown, the database splitting device may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the database splitting device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the database splitting device to communicate wirelessly or wired with other devices to exchange data. Although the figure shows a database splitting device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.

[0124] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0125] The database splitting device provided by the present application adopts the database splitting method in the above embodiment, which can solve the problem of how to split the database without affecting customer transactions. Compared with the prior art, the beneficial effects of the database splitting device provided by the present application are the same as the beneficial effects of the database splitting method provided by the above embodiment, and the other technical features in the database splitting device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0126] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0127] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0128] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the database splitting method in the above-mentioned embodiment.

[0129] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0130] The computer-readable storage medium may be included in the database splitting device; or may exist independently without being assembled into the database splitting device.

[0131] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the database splitting device, the database splitting device: when receiving a transaction request initiated by a customer in the database splitting process, determines the target sub-library based on the transaction object number of the transaction request; distributes the transaction request to the target sub-library, and stores the data generated by the transaction request in the target sub-library; after completing the transaction request distribution, determines the historical data of the old database, and transplants the historical data to the corresponding sub-libraries according to the transaction object number of the historical data.

[0132] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0133] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0134] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0135] The readable storage medium provided in the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned database splitting method, and can solve how to split the database without affecting customer transactions. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in the present application are the same as the beneficial effects of the database splitting method provided in the above-mentioned embodiment, and will not be repeated here.

[0136] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned database splitting method when executed by a processor.

[0137] The computer program product provided by this application can solve the problem of how to split the database without affecting customer transactions. Compared with the prior art, the beneficial effects of the computer program product provided by this application are the same as the beneficial effects of the database splitting method provided by the above embodiment, which will not be repeated here.

[0138] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A database splitting method, characterized in that: The method includes: When receiving a transaction request initiated by a customer during the database splitting process, determining a target sub-database based on the transaction subject number of the transaction request; Distribute the transaction request to the target sub-database, and store the data generated by the transaction request in the target sub-database; After completing the transaction request distribution, the historical data of the old database is determined, and the historical data is transplanted to the corresponding sub-databases according to the transaction subject number of the historical data.

2. The method according to claim 1, characterized in that The step of determining the target sub-database based on the transaction subject number of the transaction request includes: Parsing the transaction request to obtain information of the transaction object, and generating a number of the transaction object according to the information of the transaction object; Reading a preset routing strategy, and determining the target sub-database according to the transaction subject number and the routing strategy; Distribute the transaction request to the target sub-database.

3. The method according to claim 1, characterized in that The step of determining the target sub-database based on the transaction subject number of the transaction request further includes: When receiving a transfer target request initiated by the transferor, parsing the transfer target request to obtain information about the transfer target; Generating the transaction subject number based on the information of the transfer subject; Read the preset routing strategy, and determine the target sub-database according to the transaction subject number and the routing strategy; Distribute the transfer target request to the target sub-library.

4. The method according to claim 3, characterized in that After the step of distributing the transfer target request to the target sub-database, the method further includes: When receiving a purchase request initiated by a purchaser, parsing the purchase request to obtain a transaction target number; Reading the preset routing strategy, and determining the target sub-database according to the transaction subject number and the routing strategy; Distribute the purchase request to the target sub-database.

5. The method according to claim 1, characterized in that Before the step of determining the target sub-database based on the transaction subject number of the transaction request, the method further includes: Configure the sub-library rules and establish a mapping relationship between each sub-library and the transaction target number.

6. The method according to claim 1, characterized in that The steps of determining the historical data of the old database and transplanting the historical data to the corresponding sub-databases according to the preset transaction subject numbers of the historical data include: Acquire the transaction target information of the historical data, and generate the transaction target number of the historical data according to the transaction target information of the historical data; Read the sub-library rule, and determine the sub-library that has a mapping relationship with the transaction subject number of the historical data according to the sub-library rule and the transaction subject number of the historical data; The historical data is transplanted to the sub-database that has a mapping relationship with the transaction subject number of the historical data.

7. The method according to any one of claims 1 to 6, characterized in that The method further includes: Monitor the operation status of each of the sub-libraries, and when an abnormality occurs in a first sub-library, obtain information about the first sub-library and a transaction target number that is mapped to the first sub-library; Determine a second sub-database, and establish a new mapping relationship between the second sub-database and the transaction subject number that has a mapping relationship with the first sub-database; Update the routing strategy based on the new mapping relationship; When receiving a transaction request from a customer, the transaction request is forwarded to the second data sub-database according to the updated routing strategy.

8. A database splitting device, characterized in that: The device comprises: A target sub-database determination module, configured to determine a target sub-database based on a transaction subject number of a transaction request when receiving a transaction request initiated by a customer during the database splitting process; A transaction request distribution module, used to distribute the transaction request to the target sub-database, and store the data generated by the transaction request in the target sub-database; The historical data migration module is used to determine the historical data of the old database after completing the transaction request distribution, and migrate the historical data to the corresponding sub-databases according to the transaction target number of the historical data.

9. A database splitting device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the database splitting method according to any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the database splitting method according to any one of claims 1 to 7 are implemented.

Citation Information

Patent Citations

  • Database division method, device and transaction request data read-write system

    CN111930741A

  • Dynamically splitting multi-tenant databases

    CN102436485A

  • Splitting and moving ranges in a distributed system

    CN109074362A

  • Data processing method and device

    CN110716961A

  • Data synchronization method and device, electronic equipment and storage medium

    CN111782721A

Cited By

  • Fund registration and transfer method and system based on distributed transaction mechanism

    CN120258997A