Implementation method and related equipment of high-availability database system

Abstract

The present invention relates to the field of database technology, and discloses a method for implementing a high-availability database system and related equipment. The method comprises: detecting in real time whether a fault occurs during the operation of the computing node; when a fault occurs during the operation of the computing node, selecting a cloud host from the cloud host resource pool as a new computing node; mounting the distributed storage to the new computing node, and starting a new database instance on the new computing node; performing a rapid crash recovery operation on the database instance, and using the new computing node to provide database services externally when performing the rapid crash recovery operation. The present invention implements a stand-alone high-availability database system without the need for a master-slave architecture, thereby reducing the operating costs of the database.

Description

Technical Field

[0001] The present invention relates to the field of database technology, and in particular to an implementation method of a high-availability database system and related equipment. Background Art

[0002] A database system's entire hardware and software stack is susceptible to both hardware and software failures. In a stand-alone database system, system failures take a long time to resolve, resulting in prolonged database service unavailability. Therefore, stand-alone database systems cannot meet the application requirements of enterprise-level databases.

[0003] High-availability database systems are one of the mainstream solutions for enterprise-level databases on the market. They work by automatically handling and quickly recovering from database failures through redundant database nodes and automatic disaster recovery mechanisms. Even if a node in the database system becomes unavailable due to a problem, the database system can still provide normal database services to the outside world as a whole.

[0004] In recent years, the emergence of compute and storage separation technology on cloud computing platforms has decoupled the compute and storage layers, enabling the pooling of compute and storage resources, further enhancing the elasticity and disaster recovery capabilities of cloud resources. For example, cloud host products that now implement compute and storage separation can persist data that previously required persistence on local hard drives to remote high-performance cloud disks, transforming the cloud host into a stateless compute node (consisting solely of CPU and memory). This separation of compute and storage significantly enhances the disaster recovery capabilities of cloud hosts. Summary of the Invention

[0005] The main purpose of the present invention is to provide a method for implementing a high-availability database system and related equipment, aiming to solve the technical problem of high operating costs of traditional high-availability databases.

[0006] A first aspect of the present invention provides a method for implementing a high-availability database system, the high-availability database system comprising a computing node, a distributed storage, and at least one cloud host resource pool, the cloud host resource pool comprising multiple cloud hosts, each of the cloud hosts having the same physical configuration specifications as the computing node, the method for implementing the high-availability database system comprising:

[0007] Detect in real time whether a failure occurs during the operation of the computing node;

[0008] When a failure occurs during the operation of the computing node, a cloud host is selected from the cloud host resource pool as a new computing node;

[0009] Mounting the distributed storage to the new computing node and starting a new database instance on the new computing node;

[0010] A fast crash recovery operation is performed on the database instance, and when the fast crash recovery operation is performed, the new computing node is used to provide database services externally.

[0011] Optionally, in a first implementation of the first aspect of the present invention, performing a fast crash recovery operation on the database instance includes:

[0012] Loading unapplied redo logs from the distributed stored redo log files, parsing the unapplied redo logs, and storing the parsing results in a preset hash table;

[0013] Reading a physical page corresponding to preset necessary data from the distributed storage into the memory of the new computing node for version update, and writing the updated physical page into the distributed storage, wherein the necessary data includes: data dictionary metadata, maximum transaction ID, and globally unique ID;

[0014] Performing version recovery on the necessary data based on the updated physical page and the updated rollback log in the physical page, in units of DDL operations, so that the version of the necessary data remains consistent with that at the time when the computing node fails, wherein after the version of the necessary data is recovered, the high-availability database system provides external services;

[0015] According to the updated physical page and the updated rollback log in the physical page, version recovery is performed on the table record of the high-availability database system in units of transactions, so that the version of the table record is consistent with that when the computing node fails.

[0016] Optionally, in a second implementation of the first aspect of the present invention, when performing the fast crash recovery operation, using the new computing node to provide external database services includes:

[0017] When performing the fast crash recovery operation, receiving a database service request sent by an external client through the SQL engine where the new computing node is located, and determining a target table record to be processed by the database service request and a target physical page containing the target table record;

[0018] Initiate a first acquisition request for the target table record to a table record engine through the SQL engine, and initiate a second acquisition request for the target physical page containing the target table record to a physical page engine through the table record engine;

[0019] If the target physical page does not exist in the physical page cache pool of the physical page engine, initiating a third acquisition request for the target physical page to the distributed storage through the physical page engine;

[0020] When the distributed storage responds to the third acquisition request and returns the target physical page, intercepting the target physical page for version update, and storing the target physical page after the version update in the physical page cache pool;

[0021] Reading the target physical page from the physical page cache pool by the physical page engine and returning the target physical page to the table record engine in response to the second acquisition request;

[0022] Determining, by the table record engine, whether there is an unprocessed pending transaction in the target table record to be read in the target physical page;

[0023] If there is no unprocessed pending transaction, the target table record is returned to the SQL engine through the table record engine to respond to the first acquisition request; if there is an unprocessed pending transaction, the corresponding rollback log is loaded, and the unprocessed pending transaction is rolled back or committed using the loaded rollback log, and the target table record is returned to the SQL engine through the table record engine to respond to the first acquisition request;

[0024] Based on the target table record, the database service request is processed by the SQL engine, and the processing result is returned to the external client.

[0025] Optionally, in a third implementation of the first aspect of the present invention, the loading of unapplied redo logs from the distributedly stored redo log files, parsing the unapplied redo logs, and storing the parsing results in a preset hash table includes:

[0026] When the new computing node starts a new database instance, loading unapplied redo logs from the redo log files in the distributed storage;

[0027] Parsing the unapplied redo log to obtain a physical page number of the unapplied redo log;

[0028] A key-value pair consisting of the physical page number and the redo log is constructed with the physical page number as a keyword and the redo log corresponding to the physical page of the physical page number as a value, and the key-value pair is inserted into a preset hash table.

[0029] Optionally, in a fourth implementation manner of the first aspect of the present invention, in the hash table, redo logs with the same physical page number are stored in the same one-way linked list.

[0030] Optionally, in a fifth implementation of the first aspect of the present invention, when the distributed storage responds to the third acquisition request and returns the target physical page, intercepting the target physical page for version update, and storing the target physical page after the version update in the physical page cache pool includes:

[0031] When the distributed storage responds to the third acquisition request and returns the target physical page, intercepting the target physical page;

[0032] Searching the hash table according to the physical page number of the target physical page to obtain unapplied redo logs in the target physical page;

[0033] According to the unapplied redo log in the target physical page, the target physical page is updated in version, and the target physical page after the version update is stored in the physical page cache pool.

[0034] Optionally, in a sixth implementation of the first aspect of the present invention, the method for implementing the high-availability database system further includes:

[0035] After the high-availability database system provides external services, the physical pages of other data except the necessary data are updated according to the redo logs of the physical pages corresponding to the hash table, so that the physical page versions of the other data are consistent with those when the computing node fails.

[0036] A second aspect of the present invention provides an implementation device for a high-availability database system, the high-availability database system comprising a computing node, a distributed storage, and at least one cloud host resource pool, the cloud host resource pool comprising a plurality of cloud hosts, each of the cloud hosts having the same physical configuration specifications as the computing node, the implementation device for the high-availability database system comprising:

[0037] A detection module is used to detect in real time whether a fault occurs during the operation of the computing node;

[0038] A selection module is used to select a cloud host from the cloud host resource pool as a new computing node when a failure occurs during the operation of the computing node;

[0039] A switching module, configured to mount the distributed storage to the new computing node and start a new database instance on the new computing node;

[0040] The service module is used to perform a fast crash recovery operation on the database instance, and adopt the new computing node to provide database services externally when performing the fast crash recovery operation.

[0041] Optionally, in a first implementation of the second aspect of the present invention, the service module includes:

[0042] A crash recovery unit is used to load unapplied redo logs from the redo log files of the distributed storage, parse the unapplied redo logs, and store the parsing results in a preset hash table; read the physical pages corresponding to the preset necessary data from the distributed storage to the memory of the new computing node for version update, and write the physical pages after the version update to the distributed storage, wherein the necessary data include: data dictionary metadata, maximum transaction ID, and global unique ID; based on the updated physical pages and the updated rollback logs in the physical pages, the necessary data are version restored in units of DDL operations to ensure that the version of the necessary data is consistent with that when the computing node fails, wherein after the necessary data is version restored, the high-availability database system provides services to the outside world; based on the updated physical pages and the updated rollback logs in the physical pages, the table records of the high-availability database system are version restored in units of transactions to ensure that the version of the table records is consistent with that when the computing node fails.

[0043] Optionally, in a second implementation of the second aspect of the present invention, the service module further includes:

[0044] A service unit is configured to receive, through the SQL engine where the new computing node is located, a database service request sent by an external client when executing the fast crash recovery operation, and determine a target table record and a target physical page containing the target table record that need to be processed by the database service request; initiate a first acquisition request for the target table record to a table record engine through the SQL engine, and initiate a second acquisition request for the target physical page containing the target table record to a physical page engine through the table record engine; if the target physical page does not exist in the physical page cache pool of the physical page engine, initiate a third acquisition request for the target physical page to the distributed storage through the physical page engine; when the distributed storage responds to the third acquisition request and returns the target physical page, intercept the target physical page for version update, and store the target physical page after the version update in the physical page engine. the physical page cache pool; reading the target physical page from the physical page cache pool by the physical page engine and returning it to the table record engine in response to the second acquisition request; judging by the table record engine whether there is an unprocessed pending transaction in the target table record to be read in the target physical page; if there is no unprocessed pending transaction, returning the target table record to the SQL engine by the table record engine in response to the first acquisition request; if there is an unprocessed pending transaction, loading the corresponding rollback log, and rolling back or committing the unprocessed pending transaction by using the loaded rollback log, returning the target table record to the SQL engine by the table record engine in response to the first acquisition request; processing the database service request by the SQL engine based on the target table record, and returning the processing result to the external client.

[0045] Optionally, in a third implementation of the second aspect of the present invention, the crash recovery unit is further configured to:

[0046] The unsynchronized redo log is parsed to obtain a physical page number of the unsynchronized redo log; a key-value pair consisting of the physical page number and the redo log is constructed using the physical page number as a keyword and the redo log corresponding to the physical page of the physical page number as a value, and the key-value pair is inserted into a preset hash table.

[0047] Optionally, in a fourth implementation of the second aspect of the present invention, the service unit is further configured to:

[0048] When the distributed storage responds to the third acquisition request and returns the target physical page, the target physical page is intercepted; according to the physical page number of the target physical page, the hash table is retrieved to obtain the unapplied redo log in the target physical page; according to the unapplied redo log in the target physical page, the target physical page is version updated, and the target physical page after the version update is stored in the physical page cache pool.

[0049] Optionally, in a fifth implementation of the second aspect of the present invention, in the hash table, redo logs with the same physical page number are stored in the same one-way linked list.

[0050] Optionally, in a sixth implementation of the second aspect of the present invention, the crash recovery unit is further configured to:

[0051] After the high-availability database system provides external services, the physical pages of other data except the necessary data are updated according to the redo logs of the physical pages corresponding to the hash table, so that the physical page versions of the other data are consistent with those when the computing node fails.

[0052] A third aspect of the present invention provides a computer device, comprising: a memory and at least one processor, wherein instructions are stored in the memory;

[0053] The at least one processor calls the instructions in the memory to enable the computer device to execute the above-mentioned implementation method of the high-availability database system.

[0054] A fourth aspect of the present invention provides a computer-readable storage medium having instructions stored thereon, which, when executed by a processor, implement the above-mentioned method for implementing a high-availability database system.

[0055] The present invention provides an implementation method and related equipment for a high-availability database system. The database system of the present invention includes a computing node, a distributed storage, and at least one cloud host resource pool. When a computing node fails during operation, a cloud host is selected from the cloud host resource pool as a new computing node; the distributed storage is mounted to the new computing node, and a new database instance is started on the new computing node; a rapid crash recovery operation is performed on the database instance, and when the rapid crash recovery operation is performed, the new computing node is used to provide database services to the outside world. The present invention not only ensures that a new computing node is quickly taken over after a computing node fails and that database services are normally provided to the outside world, but also, because a cloud host in the cloud host resource pool is used as a new computing node, there is no need to adopt a master-slave architecture, thereby reducing the operating costs of the database system. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 A flowchart of an embodiment of a method for implementing a high-availability database system of the present invention is provided;

[0057] Figure 2 This is a technical architecture diagram of an embodiment of a high-availability database system of the present invention;

[0058] Figure 3 This is a schematic diagram of the implementation process of a high-availability database system according to an embodiment of the present invention;

[0059] Figure 4 This is a flow chart of an embodiment of a method for implementing a high-availability database system of the present invention for performing a rapid crash recovery operation on a database instance;

[0060] Figure 5 This is a flow chart of an embodiment of a method for implementing a high-availability database system of the present invention for providing external database services when performing a fast crash recovery operation;

[0061] Figure 6 A schematic diagram of functional modules of an embodiment of an implementation device of a high-availability database system of the present invention;

[0062] Figure 7 The figure is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0063] An embodiment of the present invention provides an implementation method and related equipment for a high-availability database system. The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or that are inherent to these processes, methods, products or devices.

[0064] It should be noted that the implementation method of the high-availability database in the present invention is applicable to all types of database systems in principle, such as MySQL, PGSQL, MongoDB, etc. In different types of database systems, the basic concepts of the database may have some differences, which will not be introduced in detail here.

[0065] Before introducing the embodiments of the present invention, some basic concepts in database technology are introduced using the MySQL database system as an example:

[0066] Table: A database system stores the basic structure of the same type of data. A table stores multiple records, each of which consists of several fields.

[0067] Physical page: A data object that stores table records. The size of a physical page is fixed (e.g., 16KB). A physical page can store multiple records from a table.

[0068] Page Buffer Pool: A physical page buffer pool maintained in memory by the database system.

[0069] Transaction: A database transaction contains multiple SQL statements. At the same time, the database transaction mechanism will ensure that these SQL statements are either all executed correctly (if all SQL statements in the transaction are executed, the transaction is successfully committed) or none of them are executed (if some SQL statements in the transaction are not executed successfully, the transaction is not successfully committed or the transaction is rolled back).

[0070] Redo log: The database system generates a corresponding redo log for each physical page modification operation. The redo log faithfully records the transaction's modifications to the physical page.

[0071] Undolog: The database system generates a corresponding rollback log for each table record inserted, modified, or deleted. This log records the version of the table record before the transaction. After the transaction commits, these rollback logs are deleted. When a transaction is rolled back, the database system uses these rollback logs to roll back the modified table record, undoing the changes made by the transaction.

[0072] It should be noted that the rollback log is also stored in the physical page, and a physical page stores multiple rollback logs; that is, the physical page (Page) is not only a data object for storing table records (Record), but also a data object for storing rollback logs (Undolog).

[0073] SQL engine: SQL engine data is cached data that does not need to be persisted, so there is no need to restore SQL engine data during crash recovery.

[0074] Table Record Engine: The table record engine manages and maintains data called table records. During crash recovery, the table records must be restored to maintain transaction-level consistency at the time of the system failure.

[0075] Physical Page Engine: The physical page engine manages and maintains data called physical pages. During crash recovery, the physical page data must be restored to maintain consistency with the time of the system failure.

[0076] Data dictionary: The data managed and maintained by the data dictionary is the metadata of the library and table indexes. The metadata itself exists in the form of table records and is stored in physical pages.

[0077] For ease of understanding, a detailed description is given below in conjunction with the embodiments of the present invention.

[0078] The high-availability database system in the embodiment of the present invention includes a computing node, a distributed storage and at least one cloud host resource pool. The cloud host resource pool includes multiple cloud hosts, and the physical configuration specifications of each cloud host are the same as those of the computing node.

[0079] See also Figure 1 and Figure 2 , Figure 1 This is a flow chart of an embodiment of a method for implementing a high-availability database system according to the present invention. Figure 2 The following is a schematic diagram of an embodiment of a high-availability database system of the present invention; in this embodiment, a method for implementing a high-availability database system includes the following steps:

[0080] S10: Detect in real time whether a fault occurs during the operation of the computing node;

[0081] When the high-availability database system is operating normally, the computing nodes receive and process the client's read and write requests, and write physical pages (Page) and redo logs (Redolog) to the distributed storage.

[0082] When a computing node of a high-availability database system fails, the computing node is in an unavailable state due to the failure, and data writing and reading to the distributed storage are stopped (writing to Page and Redolog is stopped at this time).

[0083] S20: When a failure occurs during the operation of the computing node, a cloud host is selected from the cloud host resource pool as a new computing node;

[0084] When a computing node fails, the disaster recovery management and control system of the high-availability database system detects the failure event and selects a cloud host with the same configuration as the failed computing node from the cloud host resource pool as a new computing node and deploys it.

[0085] S30: Mounting the distributed storage to the new computing node, and starting a new database instance on the new computing node;

[0086] After selecting a cloud host from the cloud host resource pool and deploying it as a new computing node, the distributed storage is switched from the failed computing node to the new computing node, and a new database instance is deployed and started on the new computing node.

[0087] S40: Performing a fast crash recovery operation on the database instance, and using the new computing node to provide database services externally when performing the fast crash recovery operation.

[0088] See also Figure 3 , Figure 3 This is a schematic diagram of the implementation process of the high-availability database system of the present invention. The implementation process of the high-availability database system of the present invention is divided into three stages: ① The high-availability database system operates normally; ② The high-availability database system fails abnormally and a new computing node is deployed; ③ The new computing node mounts distributed storage and provides database services externally.

[0089] Specifically, ① represents the normal operation of a stand-alone high-availability database system. At this point, the compute node receives and processes read and write requests from clients, writing physical pages and redo logs to distributed storage. ② represents the state of a high-availability database system experiencing an abnormal failure. Due to the failure, the compute node becomes unavailable and stops writing and reading data to and from distributed storage (it stops writing pages and redo logs). Simultaneously, the high-availability database system's disaster recovery management system detects the failure and selects a cloud host from the cloud host resource pool with the same configuration as the failed compute node as a new compute node, deploying it. ③ represents the selection of a cloud host from the cloud host resource pool and deploying it as the new compute node. Distributed storage is then switched from the failed compute node to the new compute node, and a new database instance is deployed and started on the new compute node. By performing rapid crash recovery on the database instance, the new compute node can now provide database services externally.

[0090] The present invention provides an implementation method of a high-availability database system, which can realize a single-machine high-availability database system. When a computing node fails during operation, a cloud host is selected from a cloud host resource pool as a new computing node; distributed storage is mounted to the new computing node, and a new database instance is started on the new computing node; a fast crash recovery operation is performed on the database instance, and when the fast crash recovery operation is performed, the new computing node is used to provide database services to the outside world. This not only ensures that a new computing node is quickly taken over after a computing node fails and database services are normally provided to the outside world, but also, since a cloud host in the cloud host resource pool is used as a new computing node, there is no need to adopt a master-slave architecture, thereby reducing the operating costs of the database.

[0091] See also Figure 4 , Figure 4 This is a flow chart of an embodiment of a method for implementing a high-availability database system of the present invention to perform a rapid crash recovery operation on a database instance, specifically comprising the following steps:

[0092] S401: Loading unapplied redo logs from the distributed stored redo log files, parsing the unapplied redo logs, and storing the parsing results in a preset hash table;

[0093] During the operation of the high-availability database system of the present invention, when a computing node fails and becomes unavailable, there are unfinished transactions on the computing node. At this time, it is necessary to load the unapplied redo logs from the redo log files in the distributed storage, and further parse the unapplied redo logs, and then store the parsed results in a preset hash table.

[0094] Optionally, in one embodiment, step S401 specifically includes:

[0095] When the new computing node starts a new database instance, loading unapplied redo logs from the redo log files in the distributed storage;

[0096] Parsing the unapplied redo log to obtain a physical page number of the unapplied redo log;

[0097] A key-value pair consisting of the physical page number and the redo log is constructed with the physical page number as a keyword and the redo log corresponding to the physical page of the physical page number as a value, and the key-value pair is inserted into a preset hash table.

[0098] Specifically, in the preset hash table where the key-value pairs are inserted, redo logs with the same physical page number will be stored in a one-way linked list. When the physical page number of a physical page is specified, all unapplied redo logs of the physical page can be found, thereby quickly making the new computing node consistent with the computing node at the time of failure. Further combined with data recovery work, the database can maintain crash recovery while providing external services.

[0099] S402: Reading a physical page corresponding to preset necessary data from the distributed storage into the memory of the new computing node for version update, and writing the updated physical page into the distributed storage, wherein the necessary data includes: data dictionary metadata, maximum transaction ID, and globally unique ID;

[0100] Specifically, for a high-availability database system to provide normal external services—that is, for SQL read and write requests initiated by clients to be correctly processed—it relies on two conditions: necessary data, such as data dictionary metadata, maximum transaction ID, and globally unique IDs; and physical pages that are consistent at the time of the failure, including record and undolog pages. As long as these two conditions are met, the high-availability database system can provide external services.

[0101] Therefore, it is necessary to read the physical pages corresponding to the preset necessary data from the distributed storage to the memory of the new computing node for version update, and write the updated physical pages to the distributed storage.

[0102] At this point, not only are the physical pages in the distributed storage consistent with the computing nodes at the time of the failure; at the same time, the rollback log is also implicitly consistent with the time of the failure (because the rollback log is stored in the physical pages, when the physical pages are consistent, the rollback log is naturally consistent).

[0103] S403: Performing version recovery on the necessary data based on the updated physical page and the updated rollback log in the physical page, in units of DDL operations, so that the version of the necessary data remains consistent with that at the time when the computing node fails. After the version recovery of the necessary data, the high-availability database system provides external services.

[0104] Specifically, based on the updated physical pages and rollback logs in S402, the data dictionary metadata is restored, in units of DDL (Data Definition Language) operations, to maintain transactional consistency with the time of the failure. Once the data dictionary metadata is restored, the database system can perform read and write operations using the correct metadata. Once the necessary data is restored to the same consistency as at the time of the compute node failure, the highly available database system can begin providing external services.

[0105] Optionally, in one embodiment, the method for implementing a high-availability database system further includes:

[0106] S404: After the high-availability database system provides external services, the physical pages of other data except the necessary data are updated according to the redo logs of the physical pages corresponding to the hash table, so that the physical page versions of the other data are consistent with those when the computing node fails.

[0107] In this embodiment, after the high-availability database system resumes external services, it further updates the physical pages corresponding to the remaining unrecovered data based on the redo logs written to the physical pages corresponding to the physical page numbers in the hash table. Specifically, a thread group for refreshing the physical pages with redo logs is started, and the remaining unrefreshed physical pages are refreshed using the redo logs in the hash table. The refreshed physical pages are then written to the distributed storage, gradually ensuring that the physical pages in the distributed storage are consistent with those at the time of the primary database instance failure.

[0108] S405: Based on the updated physical page and the updated rollback log in the physical page, version recovery is performed on the table record of the high-availability database system in units of transactions, so that the version of the table record is consistent with that when the computing node fails.

[0109] The version recovery operation for table records in a high-availability database system can be performed asynchronously, that is, the table records can be recovered while providing services; this allows the high-availability database to provide normal external services while performing fast crash recovery operations, without having to wait for the lengthy crash recovery process to complete.

[0110] See also Figure 5 , Figure 5 This is a flow chart of an embodiment of a method for implementing a high-availability database system of the present invention for providing database services externally when performing a fast crash recovery operation, specifically including the following steps:

[0111] S411: When executing the fast crash recovery operation, receiving a database service request sent by an external client through the SQL engine where the new computing node is located, and determining a target table record to be processed by the database service request and a target physical page containing the target table record;

[0112] Specifically, when an external client sends an SQL statement (database service request) to the SQL engine, the SQL engine generates an execution plan for the statement and executes it, and determines the target table records that need to be processed and the target physical pages containing the target table records.

[0113] S412: Initiate a first acquisition request for the target table record to a table record engine through the SQL engine, and initiate a second acquisition request for a target physical page containing the target table record to a physical page engine through the table record engine;

[0114] Specifically, during the execution process, the target table record is obtained by calling the table record engine, and the target physical page is obtained by calling the physical page engine through the table record engine.

[0115] S413: If the target physical page does not exist in the physical page cache pool of the physical page engine, initiate a third acquisition request for the target physical page to the distributed storage through the physical page engine;

[0116] Specifically, when performing a fast crash recovery operation, the entire database system is in an initialization state, and the target physical page does not exist in the physical page cache pool in the physical page engine. At this time, it is necessary to send a get request to the distributed storage through the physical page engine to try to read the physical page.

[0117] S414: When the distributed storage responds to the third acquisition request and returns the target physical page, intercepting the target physical page for version update, and storing the target physical page after the version update in the physical page cache pool;

[0118] Specifically, when the distributed storage responds to the acquisition request of the physical page engine and returns the target physical page (the target physical page at this time is the version before the failure moment), it intercepts the target physical page and updates it (the target physical page at this time is the version of the computing node at the failure moment), and stores the updated target physical page in the physical page cache pool.

[0119] S415: reading the target physical page from the physical page cache pool through the physical page engine and returning the target physical page to the table record engine to respond to the second acquisition request;

[0120] Specifically, after the updated target physical page is stored in the physical page cache pool, the physical page engine reads the target physical page from the physical page cache pool and returns it to the table record engine in response to the second acquisition request in step S412.

[0121] S416: Determine, by the table record engine, whether there is any unprocessed pending transaction in the target table record to be read in the target physical page;

[0122] Specifically, the table record engine determines the target table record to be read in the target physical page;

[0123] If it is impossible to determine whether the target table record is the table record at the time of the failure, the target table record is directly returned to the SQL engine for reading and writing;

[0124] If there is an unprocessed suspended transaction in the target table record being read (this means that the table record is the table record at the time of the failure), the target table record will not be returned to the SQL engine (if the SQL engine needs to write), or the table record version before the failure will be found in the rollback log and returned to the SQL engine (if the SQL engine needs to read).

[0125] S417: If there is no unprocessed pending transaction, the target table record is returned to the SQL engine via the table record engine to respond to the first acquisition request; if there is an unprocessed pending transaction, the corresponding rollback log is loaded, and the unprocessed pending transaction is rolled back or committed using the loaded rollback log, and the target table record is returned to the SQL engine via the table record engine to respond to the first acquisition request;

[0126] S418: Based on the target table record, the database service request is processed by the SQL engine, and the processing result is returned to the external client.

[0127] Optionally, in one embodiment, step S414 specifically includes:

[0128] When the distributed storage responds to the third acquisition request and returns the target physical page, intercepting the target physical page;

[0129] Searching the hash table according to the physical page number of the target physical page to obtain unapplied redo logs in the target physical page;

[0130] According to the unapplied redo log in the target physical page, the target physical page is updated in version, and the target physical page after the version update is stored in the physical page cache pool.

[0131] Specifically, by intercepting the target physical page and searching in the hash table according to the physical page number of the target physical page, the unapplied redo log can be quickly and accurately obtained in the target physical page, and the target physical page can be further updated (updated to the version of the computing node at the time of the failure).

[0132] The implementation method of the high-availability database system in this embodiment can realize a high-availability database system that can quickly perform crash repair while providing database services to the outside world. This is different from traditional high-availability database systems that need to wait for crash recovery to complete before providing services to the outside world.

[0133] The above describes the implementation method of the high-availability database system in the embodiment of the present invention. The following describes the implementation device of the high-availability database system in the embodiment of the present invention. Figure 6 , Figure 6 This is a functional module diagram of an embodiment of a device for implementing a high-availability database system of the present invention. In this embodiment, the device for implementing a high-availability database system includes:

[0134] Detection module 401, used to detect in real time whether a fault occurs during the operation of the computing node;

[0135] The selection module 402 is configured to select a cloud host from the cloud host resource pool as a new computing node when a failure occurs during the operation of the computing node;

[0136] The switching module 403 is used to mount the distributed storage to the new computing node and start a new database instance on the new computing node;

[0137] The service module 404 is configured to use the new computing node to provide external database services when performing a fast crash recovery operation on the database instance.

[0138] Optionally, in one embodiment, the service module 404 includes:

[0139] A crash recovery unit 4041 is used to load unapplied redo logs from the redo log files of the distributed storage, parse the unapplied redo logs, and store the parsing results in a preset hash table; read the physical pages corresponding to the preset necessary data from the distributed storage to the memory of the new computing node for version update, and write the physical pages after the version update to the distributed storage, wherein the necessary data includes: data dictionary metadata, maximum transaction ID, and global unique ID; based on the updated physical pages and the updated rollback logs in the physical pages, the necessary data is version restored in units of DDL operations, so that the version of the necessary data is consistent with that when the computing node fails, wherein after the necessary data is version restored, the high-availability database system provides services to the outside world; based on the updated physical pages and the updated rollback logs in the physical pages, the table records of the high-availability database system are version restored in units of transactions, so that the version of the table records is consistent with that when the computing node fails.

[0140] The service unit 4042 is used to receive a database service request sent by an external client through the SQL engine where the new computing node is located when executing the fast crash recovery operation, and determine the target table record that needs to be processed by the database service request and the target physical page containing the target table record; initiate a first acquisition request for the target table record to the table record engine through the SQL engine, and initiate a second acquisition request for the target physical page containing the target table record to the physical page engine through the table record engine; if the target physical page does not exist in the physical page cache pool of the physical page engine, initiate a third acquisition request for the target physical page to the distributed storage through the physical page engine; when the distributed storage responds to the third acquisition request and returns the target physical page, intercept the target physical page The method comprises the following steps: performing a version update on the physical page, and storing the target physical page after the version update in the physical page cache pool; reading the target physical page from the physical page cache pool by the physical page engine and returning it to the table record engine in response to the second acquisition request; judging by the table record engine whether there is an unprocessed pending transaction in the target table record to be read in the target physical page; if there is an unprocessed pending transaction, loading the corresponding rollback log, and using the loaded rollback log to roll back or commit the unprocessed pending transaction, returning the target table record to the SQL engine by the table record engine in response to the first acquisition request; processing the database service request by the SQL engine based on the target table record, and returning the processing result to the external client.

[0141] Optionally, in one embodiment, the crash recovery unit 4041 is further configured to:

[0142] The unsynchronized redo log is parsed to obtain a physical page number of the unsynchronized redo log; a key-value pair consisting of the physical page number and the redo log is constructed using the physical page number as a keyword and the redo log corresponding to the physical page of the physical page number as a value, and the key-value pair is inserted into a preset hash table.

[0143] Optionally, in one embodiment, the service unit 4042 is further configured to:

[0144] When the distributed storage responds to the third acquisition request and returns the target physical page, the target physical page is intercepted; according to the physical page number of the target physical page, the hash table is retrieved to obtain the unapplied redo log in the target physical page; according to the unapplied redo log in the target physical page, the target physical page is version updated, and the target physical page after the version update is stored in the physical page cache pool.

[0145] The embodiment of the present invention realizes a stand-alone high-availability database system. When a computing node fails during operation, a cloud host is selected from the cloud host resource pool as a new computing node; distributed storage is mounted to the new computing node, and a new database instance is started on the new computing node; a fast crash recovery operation is performed on the database instance, and when the fast crash recovery operation is performed, the new computing node is used to provide database services to the outside world. This not only ensures that a new computing node is quickly taken over after a computing node fails and database services are provided normally to the outside world, but also, since a cloud host in the cloud host resource pool is used as a new computing node, there is no need to adopt a master-slave architecture, thereby reducing the operating costs of the database.

[0146] The above describes in detail the implementation device of the high-availability database system in the embodiment of the present invention from the perspective of modular functional entities. The following describes in detail the computer device in the embodiment of the present invention from the perspective of hardware processing.

[0147] Please refer to Figure 7 , Figure 7 This is a schematic diagram of the hardware structure of a computer device provided in an embodiment of the present invention. The computer device 500 may vary greatly due to different configurations or performance, and may include one or more processors (central processing units, CPU) 510 (for example, one or more processors) and a memory 520, and one or more storage media 530 (for example, one or more mass storage devices) storing application programs 533 or data 532. The memory 520 and the storage medium 530 may be temporary storage or permanent storage. The program stored in the storage medium 530 may include one or more modules (not shown in the figure), each module may include a series of instruction operations in the computer device 500. Furthermore, the processor 510 may be configured to communicate with the storage medium 530 to execute a series of instruction operations in the storage medium 530 on the computer device 500.

[0148] The computer device 500 may further include one or more power supplies 540, one or more wired or wireless network interfaces 550, one or more input and output interfaces 560, and / or one or more operating systems 531, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. It will be appreciated by those skilled in the art that Figure 7 The illustrated computer device structure does not limit the computer device and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0149] The present invention also provides a computer device, which includes a memory and a processor. The memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor executes the steps of the implementation method of the high-availability database system in the above-mentioned embodiments.

[0150] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. The computer-readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute the steps of the implementation method of the high-availability database system.

[0151] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0152] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, etc., various media that can store program code.

[0153] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions described in the above embodiments can still be modified, or some of the technical features thereof can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for implementing a high-availability database system, characterized in that: The high-availability database system includes a computing node, a distributed storage, and at least one cloud host resource pool, wherein the cloud host resource pool includes multiple cloud hosts, and the physical configuration specifications of each cloud host are the same as those of the computing node. The implementation method of the high-availability database system includes: Detect in real time whether a failure occurs during the operation of the computing node; When a failure occurs during the operation of the computing node, a cloud host is selected from the cloud host resource pool as a new computing node; Mounting the distributed storage to the new computing node and starting a new database instance on the new computing node; Performing a fast crash recovery operation on the database instance, and using the new computing node to provide database services externally while performing the fast crash recovery operation, specifically includes: When performing the fast crash recovery operation, receiving a database service request sent by an external client through the SQL engine where the new computing node is located, and determining a target table record to be processed by the database service request and a target physical page containing the target table record; Initiate a first acquisition request for the target table record to a table record engine through the SQL engine, and initiate a second acquisition request for the target physical page containing the target table record to a physical page engine through the table record engine; If the target physical page does not exist in the physical page cache pool of the physical page engine, initiating a third acquisition request for the target physical page to the distributed storage through the physical page engine; When the distributed storage responds to the third acquisition request and returns the target physical page, intercepting the target physical page for version update, and storing the target physical page after the version update in the physical page cache pool; Reading the target physical page from the physical page cache pool by the physical page engine and returning the target physical page to the table record engine in response to the second acquisition request; Determining, by the table record engine, whether there is an unprocessed pending transaction in the target table record to be read in the target physical page; If there is no unprocessed pending transaction, the target table record is returned to the SQL engine through the table record engine to respond to the first acquisition request; if there is an unprocessed pending transaction, the corresponding rollback log is loaded, and the unprocessed pending transaction is rolled back or committed using the loaded rollback log, and the target table record is returned to the SQL engine through the table record engine to respond to the first acquisition request; Based on the target table record, the database service request is processed by the SQL engine, and the processing result is returned to the external client.

2. The method for implementing a high-availability database system according to claim 1, wherein: The performing of a fast crash recovery operation on the database instance includes: Loading unapplied redo logs from the distributed stored redo log files, parsing the unapplied redo logs, and storing the parsing results in a preset hash table; Reading a physical page corresponding to preset necessary data from the distributed storage into the memory of the new computing node for version update, and writing the updated physical page into the distributed storage, wherein the necessary data includes: data dictionary metadata, maximum transaction ID, and globally unique ID; Performing version recovery on the necessary data based on the updated physical page and the updated rollback log in the physical page, in units of DDL operations, so that the version of the necessary data remains consistent with that at the time when the computing node fails, wherein after the version of the necessary data is recovered, the high-availability database system provides external services; According to the updated physical page and the updated rollback log in the physical page, version recovery is performed on the table record of the high-availability database system in units of transactions, so that the version of the table record is consistent with that when the computing node fails.

3. The method for implementing a high-availability database system according to claim 2, wherein: The step of loading unapplied redo logs from the distributed stored redo log files, parsing the unapplied redo logs, and storing the parsing results in a preset hash table includes: When the new computing node starts a new database instance, loading unapplied redo logs from the redo log files in the distributed storage; Parsing the unapplied redo log to obtain a physical page number of the unapplied redo log; A key-value pair consisting of the physical page number and the redo log is constructed with the physical page number as a keyword and the redo log corresponding to the physical page of the physical page number as a value, and the key-value pair is inserted into a preset hash table.

4. The method for implementing a high-availability database system according to claim 3, wherein: In the hash table, redo logs with the same physical page number are stored in the same one-way linked list.

5. The method for implementing a high-availability database system according to claim 3, wherein: When the distributed storage responds to the third acquisition request and returns the target physical page, intercepting the target physical page for version update, and storing the target physical page after the version update in the physical page cache pool includes: When the distributed storage responds to the third acquisition request and returns the target physical page, intercepting the target physical page; Searching the hash table according to the physical page number of the target physical page to obtain unapplied redo logs in the target physical page; According to the unapplied redo log in the target physical page, the target physical page is updated in version, and the target physical page after the version update is stored in the physical page cache pool.

6. The method for implementing a high-availability database system according to claim 2, wherein: The implementation method of the high-availability database system also includes: After the high-availability database system provides external services, the physical pages of other data except the necessary data are updated according to the redo logs of the physical pages corresponding to the hash table, so that the physical page versions of the other data are consistent with those when the computing node fails.

7. A device for implementing a high-availability database system, used in the method for implementing a high-availability database system according to any one of claims 1 to 6, characterized in that: The high-availability database system includes a computing node, a distributed storage, and at least one cloud host resource pool, wherein the cloud host resource pool includes multiple cloud hosts, and the physical configuration specifications of each cloud host are the same as those of the computing node. The implementation device of the high-availability database system includes: A detection module is used to detect in real time whether a fault occurs during the operation of the computing node; A selection module is used to select a cloud host from the cloud host resource pool as a new computing node when a failure occurs during the operation of the computing node; A switching module, configured to mount the distributed storage to the new computing node and start a new database instance on the new computing node; The service module is used to perform a fast crash recovery operation on the database instance, and adopt the new computing node to provide database services externally when performing the fast crash recovery operation.

8. A computer device, characterized in that: The computer device includes: a memory and at least one processor, wherein instructions are stored in the memory; The at least one processor calls the instructions in the memory to enable the computer device to execute the implementation method of the high-availability database system according to any one of claims 1 to 6.

9. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instructions are executed by a processor, the method for implementing a high-availability database system according to any one of claims 1 to 6 is implemented.

Images