Data partitioning methods and systems

Abstract

This application discloses a data sharding method and system, comprising: a configuration center monitoring the working status of multiple databases, and when any database is detected to be in an abnormal state, modifying the disaster recovery configuration item of that database to the first disaster recovery state; after the application system detects that the disaster recovery configuration item of any database has been modified to the first disaster recovery state, when there is data to be stored that matches any database, adding a disaster recovery identifier to the data to be stored; and a data classification system obtaining the data to be stored with the disaster recovery identifier and storing it in the disaster recovery database. By adding a disaster recovery identifier to the data to be stored that matches the database in an abnormal state through the application system, and storing the data to be stored with the disaster recovery identifier in the disaster recovery database through the data classification system, the data to be stored that matches the database in an abnormal state can also be stored normally, ensuring the normal operation of the application system without data loss or system paralysis, thus improving the reliability of data sharding.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a data partitioning method and system. Background Technology

[0002] A database is a "repository for organizing, storing, and managing data according to a data structure." It is a large, organized, shareable, and uniformly managed collection of data stored permanently in a computer. As the amount of stored data increases, not only does the database occupy more space, but the speed of data retrieval also slows down. Therefore, it is necessary to distribute the database across multiple data shards, reducing the amount of data in a single data shard and thus improving database operation performance. There are three main methods of database partitioning: database sharding, table sharding, and database-table sharding.

[0003] Database sharding and table partitioning are common database partitioning methods, and currently, database sharding and table partitioning are commonly implemented using sharding and table partitioning components. However, when using sharding and table partitioning components for partitioning, if a database crash or database anomaly occurs, only abnormal results and database error information can be returned. This strategy will cause the following problems: (1) When a database sharding crashes or a database sharding anomaly occurs, data writing will fail, resulting in some data loss. (2) Once a database sharding crashes or a database anomaly occurs, the entire system will be paralyzed until the database is recovered, which will seriously affect the high availability of the system. (3) After the database is recovered, the lost data still needs to be manually recovered according to the request log, which cannot guarantee the accuracy and security of the recovered data.

[0004] Therefore, there is an urgent need for a data sharding method that can prevent the entire system from crashing or losing data when the database crashes or malfunctions, and can automatically recover data after the database is restored, while ensuring the accuracy and security of the recovered data. Summary of the Invention

[0005] In view of this, the present invention provides a data sharding method and system, the main purpose of which is to solve the problem that when the database is down or in an abnormal state during current data sharding, the entire system is paralyzed, resulting in data loss. After the database is restored, the data needs to be manually restored, and the accuracy and security of the restored data cannot be guaranteed.

[0006] According to one aspect of this application, a data sharding method is provided, the method comprising:

[0007] The configuration center monitors the working status of multiple databases, and when it detects that any database is in an abnormal state, the configuration center modifies the disaster recovery configuration item of that database to the first disaster recovery state.

[0008] After the application system detects that the disaster recovery configuration item of any of the databases has been modified to the first disaster recovery state, when there is data to be stored that matches any of the databases, it adds a disaster recovery identifier to the data to be stored and sends it to the data classification system.

[0009] After the data classification system obtains the data to be stored with the disaster recovery identifier attached, it stores the data to be stored in a preset disaster recovery database.

[0010] According to another aspect of this application, a data sharding system is provided, comprising: a data classification system, a configuration center, and an application system.

[0011] The configuration center is used to monitor the working status of multiple databases, and when any database is detected to be in an abnormal state, the disaster recovery configuration item of that database is modified to the first disaster recovery state.

[0012] The application system is used to detect when the disaster recovery configuration item of any database is modified to the first disaster recovery state, and when there is data to be stored that matches any database, add a disaster recovery identifier to the data to be stored and send it to the data classification system.

[0013] The data classification system is used to obtain the data to be stored with the disaster recovery identifier attached, and then store the data to be stored in a preset disaster recovery database.

[0014] By employing the above technical solutions, this application provides a data sharding method and system. Upon detecting any database in an abnormal state through a configuration center, the system modifies the disaster recovery configuration of the database in the abnormal state to the first disaster recovery state. The application system adds a disaster recovery identifier to the data to be stored that matches the database in the abnormal state and sends the data with the added disaster recovery identifier to the data classification system. The data classification system stores the data with the disaster recovery identifier into the disaster recovery database, ensuring that the data to be stored that matches the database in the abnormal state can also be stored normally. The application system operates normally without waiting for a response from the database in the abnormal state, preventing data loss and system paralysis, thus improving the operability and reliability of data sharding.

[0015] This application provides a data sharding method and system that sends a data recovery notification to a data classification system through a configuration center. The data classification system extracts data from the disaster recovery database that matches the data of the recovered database when it was in an abnormal state and stores it in the recovered database. This ensures that data is stored normally when the database is in an abnormal state, and that the data stored when the database was in an abnormal state is automatically recovered after the database is recovered, without causing data loss when the database was abnormal. This eliminates the need for manual data recovery and ensures the accuracy and security of the recovered data.

[0016] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0017] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0018] Figure 1 A flowchart of a data partitioning method provided in an embodiment of this application is shown;

[0019] Figure 2 A flowchart of another data partitioning method provided in an embodiment of this application is shown;

[0020] Figure 3 This paper illustrates a database sharding flowchart for a database A exception in a database sharding method provided in an embodiment of this application.

[0021] Figure 4 This paper illustrates a database sharding flowchart during the recovery of database A, as provided in an embodiment of the data sharding method of this application.

[0022] Figure 5 This illustration shows a schematic diagram of the structural connection of a data sharding system provided in an embodiment of this application.

[0023] In the diagram: 1-Data classification system; 2-Configuration center; 3-Application system; 4-Database; 5-Disaster recovery database. Detailed Implementation

[0024] The present invention will be described in detail below with reference to the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in the embodiments of the present invention can be combined with each other.

[0025] To further illustrate the technical means and effects adopted by the present invention to achieve the intended purpose, the specific embodiments, structures, features, and effects according to the present invention will be described in detail below with reference to the accompanying drawings and preferred embodiments. In the following description, different "an embodiment" or "an embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0026] To address the problem that current database sharding systems often experience system paralysis and data loss when a database crashes or malfunctions, this application provides a database sharding method, such as... Figure 1 As shown, the method includes:

[0027] 102: The configuration center monitors the working status of multiple databases, and when it detects that any database is in an abnormal state, the configuration center modifies the disaster recovery configuration item of that database to the first disaster recovery state.

[0028] 104: After the application system detects that the disaster recovery configuration item of any database has been modified to the first disaster recovery state, when there is data to be stored that matches any database, a disaster recovery identifier is added to the data to be stored and sent to the data classification system.

[0029] 106: After the data classification system obtains the data to be stored with the disaster recovery identifier, it stores the data to be stored in the preset disaster recovery database.

[0030] Before explaining the specific data sharding methods described above, let's first briefly introduce data sharding. As the amount of data in a single database increases and the database access rate becomes higher, the time required for reading and writing the database also increases, limiting the database's read and write performance. At this point, it is necessary to optimize the database performance, such as splitting a single database into multiple data shards.

[0031] Data sharding can improve database performance and increase availability, so many methods have been developed for data sharding. One of the most common methods is to use sharding and table partitioning components.

[0032] However, when using existing database sharding components to shard the database, if a database crash or anomaly occurs, data writing will fail, resulting in some data loss. From the time the database crashes or anomalies until the database is recovered, the entire system is paralyzed, severely impacting the system's high availability.

[0033] Therefore, this application combines the data classification system with the configuration center and application system to jointly complete data sharding. First, the configuration center monitors the working status of the databases, periodically checking whether the databases can provide services normally. When any database is detected to be in an abnormal state, the disaster recovery configuration item for the database in the abnormal state is modified to the first disaster recovery state, and the modified disaster recovery configuration item is pushed to the application system.

[0034] The application system, through the configuration center, listens for the disaster recovery configuration items of databases in an abnormal state. Upon detecting that the disaster recovery configuration item of a database in an abnormal state has been modified to the first disaster recovery state, it queries for the data to be stored that matches the database in the abnormal state. This data is typically sent in the form of SQL statements. Because there is a correspondence between the database and the business system, the database sharding SDK in the application system uses this correspondence to find the data to be stored that matches the database in the abnormal state. For example, if the data to be stored is sent in the form of SQL statements, and the disaster recovery configuration item of database A has been modified to the first disaster recovery state, and database A corresponds to the product business, then among the multiple SQL statements in the product business, the system searches for SQL statements that involve database A. In the found SQL statements involving database A, a disaster recovery identifier is added to the SQL statement, and the SQL statement with the disaster recovery identifier is sent to the data classification system.

[0035] The data classification system parses the received SQL statements and determines whether there is a disaster recovery flag. If there is a disaster recovery flag, it means that the database where the data should be stored is abnormal or down. If the data is still stored in the abnormal database, it may cause system paralysis and data loss. Therefore, storing the data in the disaster recovery database does not affect the storage of the data and allows the entire system to operate normally.

[0036] This invention provides a data sharding method. Compared with the prior art, this application monitors any database in an abnormal state through a configuration center, modifies the disaster recovery configuration item of the database in the abnormal state to the first disaster recovery state, adds a disaster recovery identifier to the data to be stored that matches the database in the abnormal state, and sends the data to be stored with the added disaster recovery identifier to the data classification system. The data classification system stores the data to be stored with the disaster recovery identifier into the disaster recovery database, so that the data to be stored that matches the database in the abnormal state can also be stored normally, the application system runs normally, without waiting for the response of the database in the abnormal state, and does not cause data loss or system paralysis, thus improving the operability and reliability of data sharding.

[0037] In one specific embodiment of the present invention, the data to be stored includes data, the database corresponding to the data, and a database disaster recovery identifier corresponding to the data. After the application system detects that the disaster recovery configuration item of a database in an abnormal state has been modified to a first disaster recovery state, when it needs to send data to be stored that matches the database in an abnormal state—in other words, when it needs to send data to be stored involving a database in an abnormal state—it modifies the database disaster recovery identifier corresponding to the data to a second disaster recovery state. Normally, the application system sends data to be stored where the database disaster recovery identifier corresponding to the data is not in the second disaster recovery state. Only after detecting that the disaster recovery configuration item of a database in an abnormal state has been modified to a disaster recovery state does it send data to be stored where the database disaster recovery identifier corresponding to the data is in the second disaster recovery state.

[0038] In one specific embodiment of the present invention, such as Figure 2 As shown, the data classification system parses the data to be stored to obtain the data, the corresponding database, and the database disaster recovery identifier. If the disaster recovery identifier is in the second disaster recovery state, the database is considered to be in an abnormal state or down, and cannot provide normal service. The data is then stored in a preset disaster recovery database. If the database disaster recovery identifier is not in the second disaster recovery state, the database is considered to be normal and can provide normal service. The data classification system stores the data in the database corresponding to the data. Unlike existing methods, even if the database is in an abnormal state, the data to be stored can still be stored successfully in a timely manner without waiting for error information from the database in an abnormal state.

[0039] This invention provides a data sharding method. Compared with the prior art, this application sends data to be stored with a database disaster recovery identifier corresponding to the data and indicating that the data is in the second disaster recovery state through the application system. The data classification system stores the data to be stored in the database disaster recovery database with the database disaster recovery identifier corresponding to the data and indicating that the data is in the second disaster recovery state, so that the data matching the database in the abnormal state is stored normally, the whole system runs normally, and the operation performance and reliability of data sharding are improved.

[0040] In one embodiment, the data classification system parses the data to be stored according to database sharding rules, and obtains the data, the corresponding database, and the disaster recovery identifier of the corresponding database. Database sharding rules are database partitioning rules formulated based on factors such as the number of databases or the types of business systems.

[0041] In one embodiment, the data to be stored is sent in the form of an SQL statement. The SQL statement sent by the application system includes data and the corresponding database fields. The data classification system parses the SQL statement to obtain the data and the corresponding database fields. The database fields include the database corresponding to the data and its disaster recovery identifier. The database and its disaster recovery identifier can be directly read from the database fields, or an algorithm can be used to calculate the database fields and the result can be used to read the database and its disaster recovery identifier. Alternatively, the database fields can be decrypted using a key and the result can be used to read the database and its disaster recovery identifier.

[0042] In one specific embodiment of the present invention, the working status of the database includes an abnormal state, a recovery state, and a normal state. The data classification system acquires its communication status with all databases and pushes the acquired communication status to the configuration center. The configuration center determines the working status of each database based on the communication status.

[0043] In one embodiment, the communication status displays the connection status between the data classification system and each database. If the data classification system maintains communication connections with some databases, it indicates that these databases are functioning normally. If the data classification system loses communication connections with one or more databases, it indicates that the database is abnormal or down. If the data classification system regains communication connections with one or more databases after losing them, it indicates that the abnormal database has recovered. The configuration center determines the working status of the databases based on these communication statuses. When the configuration center determines that a database is in an abnormal state, it locates the disaster recovery configuration item for the database in the abnormal state based on the database's code and abbreviation, and modifies the disaster recovery configuration item of the database in the abnormal state to the first disaster recovery state.

[0044] In one embodiment, the data classification system determines that the database is in an abnormal state based on its communication status with the database, and sends the code and abbreviation of the database in an abnormal state to the configuration center. The configuration center modifies the disaster recovery configuration item of the database to be in the first disaster recovery state based on the code and abbreviation of the database in an abnormal state.

[0045] Taking data shard A and the data to be stored sent in the form of SQL statements as an example, the above data sharding method is explained as follows: Figure 3As shown, when data shard A crashes, the data classification system obtains its communication status with data shard A. Upon detecting the crash, it pushes relevant information about the anomaly of data shard A to the configuration center. This information includes the database's code and abbreviation. Upon receiving the crash information, the configuration center modifies the corresponding configuration items for data shard A based on its code and abbreviation, changing the disaster recovery configuration to the first disaster recovery state. The database sharding SDK in the application system listens for the modification events of the configuration items for database shard A in the configuration center. The database sharding SDK is embedded in the application system. When the database sharding SDK sends SQL statements involving data shard A, it modifies the database disaster recovery identifier corresponding to the data in the SQL statement involving data shard A to the second disaster recovery state and sends the modified SQL statement to the data classification system. When the data classification system receives the SQL statement with the database disaster recovery identifier modified to the second disaster recovery state, it sends the data in the SQL statement to the disaster recovery database.

[0046] In one instance, the data classification system obtains the communication status with the database by either probing or querying network interface objects.

[0047] Specifically, liveness detection is a common method for querying communication connections. There are various ways to implement liveness detection, and a simple example will be used here to illustrate it.

[0048] For example, the data classification system sends query information to each database at first preset time intervals. The database responds to the query information, and the data classification system identifies databases that respond more than or equal to a preset number of times within a second preset time interval as having a normal connection with the data classification system.

[0049] For example, the data classification system sends query information to each database at first preset time intervals. The database responds to the query information. The data classification system identifies databases that respond less than a preset number of times within a second preset time interval as having an abnormal connection with the data classification system.

[0050] For example, the data classification system sends query information to the database with connection failure at a first preset time interval. When the database with connection failure responds more than or equal to a preset number of times within a second preset time interval, it is considered that the connection between the database and the data classification system has been restored.

[0051] Specifically, Network Interface Object (NIO) query is also a commonly used method for testing communication status. For example, the data classification system queries the connection information between the data sub-database and the data classification system at first preset time intervals. When it finds that the database is not connected to the data classification system, it is considered that the connection between the database and the data classification system is abnormal.

[0052] For example, the data classification system queries the connection information between the database with abnormal connection and the data classification system at first preset time intervals. When the database with abnormal connection is found to be connected to the data classification system, it is considered that the connection of the database with abnormal connection has been restored.

[0053] In one specific embodiment of the present invention, when the configuration center determines that any database is in a recovery state, it modifies the disaster recovery configuration item of the recovered database to be not in the first disaster recovery state and pushes it to the application system, while simultaneously sending a data recovery notification to the data classification system; after the application system detects that the disaster recovery configuration item of the recovered database has been modified to be not in the first disaster recovery state, it modifies the database disaster recovery identifier in the data to be stored that matches the recovered database to be not in the second disaster recovery state and sends it to the data classification system; the data classification system parses the data to be stored to obtain the data, the database corresponding to the data, and the database disaster recovery identifier corresponding to the data. When the database disaster recovery identifier corresponding to the data is not in the second disaster recovery state, the data is stored in the database corresponding to the data.

[0054] This invention provides a data sharding method. Compared with the prior art, this application, through the configuration center monitoring the recovery of a database in an abnormal state, modifies the disaster recovery configuration item of the database to be in a state other than the first disaster recovery state. After the application system monitors the configuration center's modification of the disaster recovery configuration item of the database to be recovered to a state other than the first disaster recovery state, it modifies the database disaster recovery identifier corresponding to the data in the data to be stored involving the database to be recovered to a state other than the second disaster recovery state or uses the default state of not being in the second disaster recovery state. This allows the data classification system to determine that the database in an abnormal state has been recovered based on the data to be stored, without having to store and call the result of whether the database is normal or not determined based on communication information, saving storage space and calling time, making the operation of the entire system smoother and more reasonable, and improving the operating performance and reliability of data sharding.

[0055] In one embodiment, after the data classification system sends the connection status of each database to the configuration center, the configuration center determines whether the database is in an abnormal state, a recovery state, or a normal state based on the connection status. When it is determined that the database is in a recovery state, the configuration center finds the disaster recovery configuration item of the database in the recovery state based on the database's code and abbreviation, and modifies the disaster recovery configuration item of the database in the recovery state to not be in the first disaster recovery state.

[0056] In one embodiment, the data classification system determines that the database is in a recovery state based on its communication status with the database, and sends the code and abbreviation of the database in the recovery state to the configuration center. The configuration center modifies the disaster recovery configuration item of the database to not be in the first disaster recovery state based on the code and abbreviation of the database in the recovery state.

[0057] Taking data shard A and the data to be stored sent in the form of SQL statements as an example, the above data sharding method is explained as follows: Figure 4 As shown, when database shard A recovers from a crash, the data classification system detects the recovery of database shard A through its communication with it and pushes relevant information about the recovery to the configuration center. This information includes the code and abbreviation of database shard A. Upon receiving the recovery information, the configuration center, based on the code and abbreviation of database shard A, modifies the disaster recovery configuration item corresponding to database shard A to indicate that it is not in the first disaster recovery state. Simultaneously, the configuration center notifies the data classification system to begin restoring the data that should have been stored during the crash of database shard A. After the application system's sharding SDK detects the configuration change to indicate that the first disaster recovery state for database shard A is not present, it modifies the database disaster recovery flag corresponding to the data in the SQL statements involving database shard A to indicate that it is not in the second disaster recovery state. The data classification system receives this SQL statement, parses it, finds that the database disaster recovery flag corresponding to the data is not in the second disaster recovery state, and writes the data normally to database shard A.

[0058] In one embodiment, by default, the database disaster recovery flag corresponding to the data in the SQL statement is not in the second disaster recovery state. When the sharding SDK in the application system detects that the configuration item for data shard A in the configuration center has been modified to not be in the first disaster recovery state, it sends the default SQL statement involving data shard A. The database disaster recovery flag corresponding to the data in this SQL statement is not in the second disaster recovery state. The data classification system receives this SQL statement, parses it, and finds that the database disaster recovery flag corresponding to the data is not in the second disaster recovery state, and then writes the data normally to data shard A. In other words, the default database disaster recovery flag sent by the application system is not in the second disaster recovery state. When the data shard is abnormal, during the abnormal data sharding period, the application system modifies and sends the SQL statement with the abnormal database disaster recovery flag. After the abnormal data shard recovers, the default SQL statement without the disaster recovery flag is used again.

[0059] In one specific embodiment of the present invention, the data classification system extracts data matching the recovered database from the disaster recovery database and stores it in the recovered database according to the data recovery notification.

[0060] For example, such as Figure 4As shown, when database shard A recovers from a crash, the data classification system detects the recovery of database shard A through its communication with it and pushes relevant information about the recovery to the configuration center. Upon receiving the recovery message for database shard A, the configuration center modifies the corresponding disaster recovery configuration item for database shard A to indicate it is not in the first disaster recovery state, based on the database shard A's code and abbreviation. The configuration center pushes the configuration item modification information for database shard A to the application system and simultaneously notifies the data classification system to begin restoring the data that should have been stored during the crash of database shard A. Based on the notification of data recovery, the data classification system extracts the data that should have been stored during the crash of database shard A from the disaster recovery database and restores it to database shard A.

[0061] This invention provides a data partitioning method. Compared with the prior art, after a database in an abnormal state is restored, this application sends a recovery data notification to the data classification system through the configuration center. The data classification system extracts data from the disaster recovery database that matches the data when the restored database was in an abnormal state and stores it in the restored database. This ensures that data is stored normally when the database is in an abnormal state. After the database is restored, the data stored when the database was in an abnormal state is automatically restored, without causing data loss when the database was abnormal. Manual data restoration is not required, ensuring the accuracy and security of the restored data.

[0062] Furthermore, as a response to the above Figure 1 The implementation of the method shown in this invention provides a data sharding system, such as... Figure 5 As shown, the system includes: a data classification system 1, a configuration center 2, and an application system 3.

[0063] Configuration Center 2 is used to monitor the working status of multiple databases 4, and when any database 4 is detected to be in an abnormal state, the disaster recovery configuration item of any database 4 is modified to the first disaster recovery state.

[0064] Application system 3 is used to monitor when the disaster recovery configuration item of any database 4 is modified to the first disaster recovery state. When there is data to be stored that matches any database 4, it adds a disaster recovery identifier to the data to be stored and sends it to data classification system 1.

[0065] The data classification system 1 is used to obtain the data to be stored with disaster recovery identifiers and then store the data to be stored in the preset disaster recovery database 5.

[0066] This invention provides a data sharding system. Compared with existing technologies, this application, through a configuration center, detects any database in an abnormal state and modifies the disaster recovery configuration item of the database in the abnormal state to the first disaster recovery state. The application system adds a disaster recovery identifier to the data to be stored that matches the database in the abnormal state and sends the data to be stored with the added disaster recovery identifier to the data classification system. The data classification system stores the data to be stored with the disaster recovery identifier into the disaster recovery database, so that the data to be stored that matches the database in the abnormal state can also be stored normally. The application system runs normally without waiting for the response of the database in the abnormal state, without causing data loss or system paralysis, thus improving the operability and reliability of data sharding.

[0067] In one specific embodiment of the present invention, the data to be stored includes data, the database corresponding to the data, and a database disaster recovery identifier corresponding to the data. After the application system 3 detects that the disaster recovery configuration item of the database 4 in an abnormal state has been modified to the first disaster recovery state, when it needs to send the data to be stored that matches the database 4 in an abnormal state, in other words, when it needs to send the data to be stored involving the database in an abnormal state, it modifies the database disaster recovery identifier corresponding to the data to the second disaster recovery state. Normally, the application system 3 sends the data to be stored in which the database corresponding to the data is not in the second disaster recovery state. Only after detecting that the disaster recovery configuration item of the database in an abnormal state has been modified to the first disaster recovery state does it send the data to be stored in which the database disaster recovery identifier corresponding to the data is in the second disaster recovery state.

[0068] In one specific embodiment of the present invention, the data classification system 1 parses the data to be stored to obtain the data, the database corresponding to the data, and the disaster recovery identifier of the database corresponding to the data. If the disaster recovery identifier of the data is in the second disaster recovery state, it is considered that the database 4 corresponding to the data is in an abnormal state or is down, and the database cannot provide normal service. The data is then stored in a preset disaster recovery database 5. If the disaster recovery identifier of the database corresponding to the data is not in the second disaster recovery state, it is considered that the database 4 corresponding to the data is normal, and the database 4 can provide normal service. The data classification system 1 stores the data in the database 4 corresponding to the data. Unlike the existing method, even if the database is in an abnormal state, the data to be stored can be stored successfully in a timely manner without waiting for the error information of the database in an abnormal state.

[0069] This invention provides a data sharding system. Compared with the prior art, this application sends data to be stored with a database disaster recovery identifier corresponding to the data and indicating that the data is in the second disaster recovery state through the application system. The data classification system stores the data to be stored in the disaster recovery database with the database disaster recovery identifier corresponding to the data and indicating that the data is in the second disaster recovery state. This allows the data that matches the database in the abnormal state to be stored normally, and the entire system operates normally, improving the performance and reliability of data sharding.

[0070] In one embodiment, the data classification system parses the data to be stored according to database sharding rules, and obtains the data, the corresponding database, and the disaster recovery identifier of the corresponding database. Database sharding rules are database partitioning rules formulated based on factors such as the number of databases or the types of business systems.

[0071] In one specific embodiment of the present invention, the working state of database 4 includes an abnormal state, a recovery state, and a normal state. Data classification system 1 acquires its communication status with all databases 4 and pushes the acquired communication status to configuration center 2. Configuration center 2 determines the working state of each database based on the communication status.

[0072] In one embodiment, after the data classification system 1 sends the connection status of each database 4 to the configuration center 2, if the data classification system 1 maintains communication connections with some databases, it indicates that these databases are normal; if the data classification system 1 loses communication connections with one or more databases, it indicates that the database is abnormal or down; if the data classification system 1 regains communication connections with one or more databases after losing them, it indicates that the abnormal database has recovered. The configuration center 2 determines the working status of the databases based on these communication statuses. The configuration center 2 determines whether database 4 is in an abnormal state, a recovery state, or a normal state based on the connection status. When it is determined to be in an abnormal state, it finds the disaster recovery configuration item for the database in the abnormal state based on the database's code and abbreviation, and modifies the disaster recovery configuration item of the database in the abnormal state to the first disaster recovery flag.

[0073] In one embodiment, the data classification system 1 determines that the database 4 is in an abnormal state based on its communication status with the database 4, and sends the code and abbreviation of the database 4 in the abnormal state to the configuration center 2. The configuration center 2 modifies the disaster recovery configuration item of the database to be in the first disaster recovery state based on the code and abbreviation of the database in the abnormal state.

[0074] In one specific embodiment of the present invention, the data classification system 1 determines the working status of the database based on its communication status with the database 4, and sends the code and abbreviation of the database in an abnormal state or a recovery state to the configuration center 2. The configuration center 2 modifies the disaster recovery configuration item of the database in the abnormal state to be in the first disaster recovery state based on the code and abbreviation of the database in the abnormal state.

[0075] In one instance, data classification system 1 obtains the communication status with the database by either probing or querying network interface objects.

[0076] In one specific embodiment of the present invention, when the configuration center 2 determines that any database 4 is in a recovery state, it modifies the disaster recovery configuration item of the recovered database 4 to be not in the first disaster recovery state and pushes it to the application system 3, and at the same time sends a data recovery notification to the data classification system 1; after the application system 3 detects that the disaster recovery configuration item of the recovered database 4 has been modified to be not in the first disaster recovery state, it modifies the database disaster recovery identifier in the data to be stored that matches the recovered database to be not in the second disaster recovery state, and sends it to the data classification system 1; the data classification system 1 parses the data to be stored to obtain the data, the database corresponding to the data, and the database disaster recovery identifier corresponding to the data. When the database disaster recovery identifier corresponding to the data is not in the second disaster recovery state, the data is stored in the database 4 corresponding to the data.

[0077] In one embodiment, after the data classification system 1 sends the connection status of each database 4 to the configuration center 2, the configuration center 2 determines whether the database is in an abnormal state, a recovery state, or a normal state based on the connection status. When it is determined that the database is in a recovery state, the configuration center 2 finds the disaster recovery configuration item of the database 4 in the recovery state based on the code and abbreviation of the database in the recovery state, and modifies the disaster recovery configuration item of the database 4 in the recovery state to not be in the first disaster recovery state.

[0078] In one embodiment, the data classification system 1 determines that the database 4 is in a recovery state based on its communication status with the database 4, and sends the code and abbreviation of the database in the recovery state to the configuration center 2. The configuration center 2 modifies the disaster recovery configuration item of the database 4 to not be in the first disaster recovery state based on the code and abbreviation of the database 4 in the recovery state.

[0079] This invention provides a data sharding system. Compared with existing technologies, this application, through the configuration center's monitoring of a database in an abnormal state and its subsequent recovery, modifies the disaster recovery configuration item of the recovered database to be not in the first disaster recovery state. When the application system monitors the configuration center's modification of the disaster recovery configuration item of the recovered database to be not in the first disaster recovery state, it modifies the database disaster recovery identifier corresponding to the data to be stored in the database involving the recovered database to be not in the second disaster recovery state or uses the default state of not being in the second disaster recovery state. This allows the data sharding system to determine that the database in an abnormal state has been recovered based on the data to be stored, without needing to store and retrieve the result of whether the database is normal based on communication information. This saves storage space and retrieval time, making the entire system run more smoothly and rationally, and improving the performance and reliability of data sharding.

[0080] In one specific embodiment of the present invention, the data classification system 1 extracts data that matches the recovered database 4 from the disaster recovery database 5 and stores it in the recovered database 4 according to the data recovery notification.

[0081] This invention provides a data sharding system. Compared with the prior art, after a database in an abnormal state is restored, this application sends a recovery data notification to the data classification system through the configuration center. The data classification system extracts the data that should have been stored when the restored database was in an abnormal state from the disaster recovery database and stores it in the restored database. This ensures that data is stored normally when the database is in an abnormal state. Moreover, after the database is restored, the data stored when the database was in an abnormal state is automatically restored, without causing data loss when the database was abnormal. Manual data restoration is not required, ensuring the accuracy and security of the restored data.

[0082] The above embodiments are merely exemplary embodiments of this application and are not intended to limit this application. The scope of protection of this application is defined by the claims. Those skilled in the art can make various modifications or equivalent substitutions to this application within its substance and scope of protection, and such modifications or equivalent substitutions should also be considered to fall within the scope of protection of this application.

Claims

1. A data library partitioning method, characterized by, The application comprises the following steps: The configuration center monitors the working states of multiple databases, and modifies the disaster recovery configuration item of any database to a first disaster recovery state when the configuration center monitors that any database is in an abnormal state; After the application system monitors that the disaster recovery configuration item of any database is modified to the first disaster recovery state, when there is to-be-stored data matching the any database, the application system adds a disaster recovery identifier to the to-be-stored data and issues the to-be-stored data to a data classification system; After the data classification system obtains the to-be-stored data with the disaster recovery identifier, the data classification system stores the to-be-stored data in a preset disaster recovery database; The configuration center monitors the working states of multiple databases, which comprises that the configuration center obtains the connection states of each database and the data classification system from the data classification system, and determines the working state of the database according to the connection states; When the configuration center determines that any database is in a recovery state, the configuration center modifies the disaster recovery configuration item of the recovered database to not be in the first disaster recovery state and issues a data recovery notification to the data classification system; After the application system monitors that the disaster recovery configuration item of the recovered database is modified to not be in the first disaster recovery state, the application system modifies the database disaster recovery identifier in the to-be-stored data matching the recovered database to not be in a second disaster recovery state and issues the to-be-stored data to the data classification system; The data classification system analyzes the to-be-stored data, obtains data, a database corresponding to the data and a database disaster recovery identifier corresponding to the data, and stores the data in the database corresponding to the data when the database disaster recovery identifier corresponding to the data is not in the second disaster recovery state.

2. The data partitioning method of claim 1, wherein, The to-be-stored data comprises data, a database corresponding to the data and a database disaster recovery identifier corresponding to the data, and the application system adds a disaster recovery identifier to the to-be-stored data, which comprises that the application system modifies the database disaster recovery identifier corresponding to the data to be in the second disaster recovery state. After the data classification system obtains the to-be-stored data with the disaster recovery identifier, the data classification system stores the to-be-stored data in a preset disaster recovery database, which comprises that the data classification system analyzes the to-be-stored data, obtains data, a database corresponding to the data and a database disaster recovery identifier corresponding to the data, and stores the data in the preset disaster recovery database when the database disaster recovery identifier corresponding to the data is in the second disaster recovery state.

3. The data partitioning method of claim 2, wherein, The working state of the database comprises an abnormal state, a recovery state and a normal state, and the configuration center monitors the working states of multiple databases, which comprises that the data classification system obtains the communication states of the data classification system and multiple databases, and pushes the communication states to the configuration center; and the configuration center determines the working state of each database according to the communication states. The data classification system extracts data matching the recovered database from the disaster recovery database according to the data recovery notification and stores the data in the recovered database.

4. The data partitioning method of claim 2, wherein, The application comprises the following steps: The application comprises the following steps: The configuration center monitors the working states of multiple databases, and modifies the disaster recovery configuration item of any database to a first disaster recovery state when the configuration center monitors that any database is in an abnormal state; 5. The method of claim 1, wherein, ​ 6. A data partitioning system, comprising: ​ ​ ​ The application system is configured to, after listening to the disaster recovery configuration item of any database being modified to the first disaster recovery state, add a disaster recovery identifier to the to-be-stored data when there is to-be-stored data matching the any database, and issue the to-be-stored data to the data classification system; The data classification system is configured to, after obtaining the to-be-stored data with the disaster recovery identifier, store the to-be-stored data in a preset disaster recovery database; The configuration center is further configured to obtain, from the data classification system, a connection state of each database and the data classification system, and determine a working state of the database according to the connection state; When the configuration center determines that any database is in a recovery state, the configuration center modifies the disaster recovery configuration item of the recovered database to not be in the first disaster recovery state, and issues a data recovery notification to the data classification system; After the application system listens to the disaster recovery configuration item of the recovered database being modified to not be in the first disaster recovery state, the application system modifies the database disaster recovery identifier in the to-be-stored data matching the recovered database to not be in the second disaster recovery state, and issues the to-be-stored data to the data classification system; The data classification system analyzes the to-be-stored data to obtain data, a database corresponding to the data, and a database disaster recovery identifier corresponding to the data, and stores the data in the database corresponding to the data when the database disaster recovery identifier corresponding to the data is not in the second disaster recovery state.

7. The data partitioning system of claim 6, wherein, The to-be-stored data includes data, a database corresponding to the data, and a database disaster recovery identifier corresponding to the data, and the application system is configured to add a disaster recovery identifier to the to-be-stored data, including: modifying the database disaster recovery identifier corresponding to the data to be in the second disaster recovery state.

8. The data partitioning system of claim 7, wherein, After the data classification system obtains the to-be-stored data with the disaster recovery identifier, the data classification system stores the to-be-stored data in a preset disaster recovery database, including: The data classification system analyzes the to-be-stored data to obtain data, a database corresponding to the data, and a database disaster recovery identifier corresponding to the data, and stores the data in the preset disaster recovery database when the database disaster recovery identifier corresponding to the data is in the second disaster recovery state.

9. The data partitioning system of claim 7, wherein, The working state of the database includes an abnormal state, a recovery state, and a normal state, and the configuration center is configured to listen to the working state of the plurality of databases, including: The data classification system obtains a communication state of the data classification system and the plurality of databases, and pushes the communication state to the configuration center; The configuration center determines the working state of each database according to the communication state.

Images