Data backtracking method and apparatus

Abstract

The application discloses a data backtracking method and device, and belongs to the technical field of databases. The method comprises the following steps: acquiring first metadata corresponding to a target historical moment in metadata of log data; acquiring second metadata corresponding to the target historical moment in metadata of each piece of shard data in a plurality of pieces of shard data, wherein the plurality of pieces of shard data are data obtained by performing shard processing on the log data; processing the log data according to the first metadata to obtain log data of the target historical moment; and processing the plurality of pieces of shard data according to the second metadata to obtain a plurality of pieces of shard data of the target historical moment. The application can reduce the data backtracking delay. The application is used for disaster recovery of databases.

Description

[0001] This application claims priority to Chinese Patent Application No. 202010748428.X, filed on July 30, 2020, entitled "Method, Device, Server and Storage Medium for Snapshot Flashback", the entire contents of which are incorporated herein by reference. TECHNICAL FIELD

[0002] The present application relates to the technical field of database, and in particular to a data rollback method and device. BACKGROUND

[0003] In order to improve the disaster recovery of a database (DB), data rollback needs to be performed in some scenarios. For example, in some databases, a logical record of each update operation needs to be recorded in a specified storage space when the update operation, such as a data manipulation language (DML) operation, is performed. When data rollback is needed, each update operation is read from the specified storage space in chronological order from recent to remote, and the read update operation is reversed until the data is rolled back to a specified time point. However, the rollback process is complex, and the data rollback delay is long. SUMMARY

[0004] Embodiments of the present application provide a data rollback method and device. The technical solution is as follows:

[0005] In a first aspect, a data rollback method is provided, all or part of which can be executed by a database system, for example, by a management node. The method includes:

[0006] obtaining first metadata corresponding to a target historical time in metadata of log data; obtaining second metadata corresponding to the target historical time in metadata of each piece of sharded data in a plurality of pieces of sharded data, the plurality of pieces of sharded data being data obtained by sharding the log data; processing the log data according to the first metadata to obtain log data at the target historical time; and processing the plurality of pieces of sharded data according to the second metadata to obtain a plurality of pieces of sharded data at the target historical time.

[0007] In embodiments of the present application, the log data is rolled back based on the first metadata corresponding to the target historical time of the log data, and the sharded data is rolled back based on the second metadata corresponding to the target historical time of the sharded data. The rollback efficiency of the data is high, and the data rollback delay is effectively reduced without reading logical records and reversing update operations.

[0008] In an optional manner, the first metadata corresponding to the target historical moment and the second metadata corresponding to the target historical moment are acquired through snapshots. The acquisition of the first metadata and the second metadata through snapshots can improve the speed of metadata acquisition and reduce the acquisition delay.

[0009] In an optional manner, the process of processing the log data according to the first metadata to obtain log data of a target historical moment includes: rolling back the log data according to the data position indicated by the first metadata to obtain log data of the target historical moment.

[0010] In another optional manner, the process of processing the plurality of shard data according to the second metadata to obtain a plurality of shard data of a target historical moment includes: rolling back the plurality of shard data according to the data position indicated by the second metadata to obtain a plurality of shard data of the target historical moment.

[0011] Due to the inconsistency of the aforementioned collection time or other reasons, the log data of the target historical moment may not be consistent with the plurality of shard data of the target historical moment (i.e., part of the data in the log data does not fall into the corresponding shard), and the data rollback method provided by the embodiments of the present application further includes: updating the plurality of shard data of the target historical moment based on the log data of the target historical moment. In this way, the log data of the target historical moment and the shard data of the target historical moment can be consistent.

[0012] In an example, the updating process includes: updating the plurality of shard data of the target historical moment based on shard relationship data and the log data of the target historical moment, the shard relationship data being used to record the correspondence between the sub-data in the log data and the shard.

[0013] Since the management node determines the shard data corresponding to the log data based on the shard relationship data, the shard relationship data also needs to be obtained when the shard data is obtained. In an optional manner, the shard relationship data corresponding to each collection time point can be directly stored, and the shard relationship data is extracted when data is traced back. In another optional manner, the shard relationship rule can be stored to obtain the shard relationship data based on the shard relationship rule when data is traced back. In still another optional manner, the shard relationship data has metadata, that is, the data corresponding to the shard relationship includes the shard relationship data and third metadata, and the management node can store the third metadata to obtain the shard relationship data based on the metadata when data is traced back. The third metadata is used to describe the shard relationship data. When data is traced back, the shard relationship data is recovered based on the third metadata of the shard relationship data. The shard relationship data is obtained based on the third metadata, and the shard relationship data can be quickly obtained. In this way, only the third metadata needs to be stored when data is not traced back, and the shard relationship data is recovered based on the third metadata when the shard relationship data is needed, thereby reducing the occupation of the storage space.

[0014] The process of updating the plurality of shard data based on the shard relationship data and the log data of the target historical time point includes: determining the incremental log data based on the data end position indicated by the first metadata of the log data and the data end position indicated by the second metadata of the plurality of shard data; determining at least one incremental shard data based on the incremental log data and the shard relationship data; and updating the plurality of shard data based on the shard relationship data and the at least one incremental shard data, and the updated plurality of shard data includes the at least one incremental shard data.

[0015] In an example, the updating process of the incremental shard data in the embodiment of the present application supports the idempotent feature. Then, the process of updating the plurality of shard data based on the shard relationship data and the at least one incremental shard data includes:

[0016] For each incremental shard data, it is detected whether the data stored in the corresponding target shard is the same as the incremental shard data; when the data stored in the corresponding target shard is different from the incremental shard data, the incremental shard data is written into the corresponding target shard; when the data stored in the corresponding target shard is the same as the incremental shard data, the incremental shard data is prohibited to be written, or the incremental shard data is used to overwrite the data stored in the corresponding target shard that is the same as the incremental shard data. The target shard is the shard corresponding to the incremental shard data based on the shard relationship data. In the above two ways, it can be avoided that the same data exists in the finally stored shard data, the occupation of the storage space by the redundant data is reduced, and the idempotent feature is realized.

[0017] For example, the first metadata of the log data indicates a data end position represented by a continuous disk writing LSN, which is the largest LSN among LSNs corresponding to log data continuously written at a corresponding time; and the second metadata of the plurality of piece data indicates a data end position represented by a global piece disk writing LSN, which is the largest LSN among LSNs corresponding to piece data globally continuously written at a corresponding time.

[0018] Optionally, the metadata of any of the log data and the piece data comprises: data writing information of each of at least one file used to store the any data; and the data writing information of any file is used to record a position of the any file in the at least one file and an end position of written data (i.e., the any data written) in the any file. Through such metadata, the position of each file can be effectively identified, so that the files are sequentially and continuously acquired when data is traced back, the continuity of the files is ensured, and valid any data is recovered.

[0019] Since the LSN represents the total amount of bytes of a transaction written to a transaction log, it is obvious that the LSN is typical data amount indication information for the transaction log. Since log data in a file is continuously written, as the number of transaction records written increases, the LSN also gradually increases. Therefore, the largest LSN (i.e., the LSN of the most recently written transaction record) carried by the written data in the file can reflect the end position of the written data. Optionally, the data writing information of the any file comprises: the largest LSN carried by the written data in the any file, so as to identify the end position of the written data. Optionally, the data writing information of any file corresponding to the log data further comprises: a file position identifier of the any file. The file position identifier is used to identify the position of the any file in the at least one file to which the any file belongs.

[0020] In an optional manner, the first metadata and the second metadata corresponding to the target historical time are obtained by screening metadata corresponding to k collection times, where k is a positive integer, and the metadata corresponding to each of the k collection times includes metadata of log data and metadata of a plurality of shard data. In this way, the metadata corresponding to each collection time is actually a set of metadata. For example, the management node can periodically collect metadata of the database or collect metadata of the database after receiving a data collection instruction to obtain metadata corresponding to k collection times. The collection time can be a collection start time, a collection end time, or a time between the collection start time and the collection end time (i.e., a time within a metadata collection period), which is used to identify the state of each metadata in the database within the metadata collection period. It should be noted that if the collection period is long, the state of each metadata in the database within the metadata collection period can be accurately identified by using the collection end time as the aforementioned collection time because the data can change from the collection start time to the collection end time.

[0021] In another optional manner, the first metadata corresponding to the target historical time is obtained by screening metadata corresponding to n first collection times, where n is a positive integer, and the metadata corresponding to each of the n collection times includes metadata of log data; and the second metadata corresponding to the target historical time is obtained by screening metadata corresponding to m second collection times, where m is a positive integer, and the metadata corresponding to each of the m collection times includes metadata of a plurality of shard data.

[0022] Optionally, m is equal to n, the n first collection times and the m second collection times correspond to each other, and a time difference between the corresponding first collection time and the second collection time is less than a difference threshold. By setting the first collection time and the second collection time to be close or the same, the process of metadata collection can be simplified, and data backtracking can be facilitated.

[0023] In a second aspect, a data backtracking apparatus is provided, which includes at least one module configured to implement the data backtracking method provided in the first aspect or any possible implementation of the first aspect.

[0024] In a third aspect, a computer device is provided, which includes a processor and a memory. The memory stores computer instructions, and the processor executes the computer instructions stored in the memory, so that the computer device executes the method provided in the first aspect or any possible implementation of the first aspect, and so that the computer device deploys the data backtracking apparatus provided in the second aspect or any possible implementation of the second aspect.

[0025] In a fourth aspect, the present application provides a computer readable storage medium, which stores computer instructions. The computer instructions instruct a computer device to perform the method provided by the first aspect or any possible implementation of the first aspect, or instruct the computer device to deploy the data rollback apparatus provided by the second aspect or any possible implementation of the second aspect.

[0026] In a fifth aspect, the present application provides a computer program product, which includes computer instructions stored in a computer readable storage medium. A processor of a computer device can read the computer instructions from the computer readable storage medium. The processor executes the computer instructions, so that the computer device performs the method provided by the first aspect or any possible implementation of the first aspect, or so that the computer device deploys the data rollback apparatus provided by the second aspect or any possible implementation of the second aspect.

[0027] In a sixth aspect, a chip is provided. The chip can include programmable logic circuitry and / or program instructions, which are used to implement the data rollback method according to any one of the first aspect when the chip is running.

[0028] In summary, in the embodiments of the present application, the first metadata is used to rollback the log data corresponding to the target historical time, and the second metadata is used to rollback the sharded data corresponding to the target historical time. Without performing the inverse operation of the logical record reading and updating operation, the data rollback efficiency is higher, and the data rollback delay is effectively reduced. BRIEF DESCRIPTION OF DRAWINGS

[0029] Figure 1 is a schematic diagram of a generation principle of sharded data provided by the embodiments of the present application;

[0030] Figure 2 is a schematic diagram of an application environment of a distributed database system involved in the data rollback method provided by the embodiments of the present application;

[0031] Figure 3 is a schematic diagram of another application environment of a distributed database system involved in the data rollback method provided by the embodiments of the present application;

[0032] Figure 4 is a schematic diagram of a structure of a transaction record provided by the embodiments of the present application;

[0033] Figure 5 is a schematic diagram of a flow of a data rollback method provided by the embodiments of the present application;

[0034] Figure 6 is a schematic diagram of a flow of another data rollback method provided by the embodiments of the present application;

[0035] Figure 7 is a time axis schematic diagram of a data backtracking method provided by an embodiment of the present application;

[0036] Figure 8 is a time axis schematic diagram of another data backtracking method provided by an embodiment of the present application;

[0037] Figure 9 is a schematic storage state diagram of log data of a target historical moment provided by an embodiment of the present application;

[0038] Figure 10 is a schematic storage state diagram of sharding data of a target historical moment provided by an embodiment of the present application;

[0039] Figure 11 is a backtracking principle schematic diagram provided by an embodiment of the present application;

[0040] Figure 12 is a structure schematic diagram of a database system provided by an embodiment of the present application;

[0041] Figure 13 is a rollback effect schematic diagram provided by an embodiment of the present application;

[0042] Figure 14 is a structure schematic diagram of a data backtracking device provided by an embodiment of the present application;

[0043] Figure 15 is a structure schematic diagram of a data backtracking device provided by an embodiment of the present application;

[0044] Figure 16 A possible basic hardware architecture of a computer device provided by an embodiment of the present application is schematically provided. DETAILED DESCRIPTION

[0045] In order to make the principles and technical solutions of the present application clearer, the following will further describe the embodiments of the present application in detail with reference to the drawings.

[0046] In order to facilitate the reader to understand, the following explains the terms involved in the embodiments of the present application:

[0047] Relational database: refers to a database that uses a relational model to organize data, which stores data in the form of rows and columns. Each relational model can be referred to as a relational table. According to different storage principles, relational databases can be divided into distributed relational databases and non-distributed relational databases.

[0048] Transaction: is a unit to maintain the consistency and integrity of database data, which has the characteristics of "all success or all failure". A transaction is not responsible for the consistency and integrity of its data before it is committed. But once the transaction is committed, the database must guarantee the consistency and integrity of the data, simply speaking, the transaction cannot lose data after it is committed. In a relational database, in order to improve performance, the transaction data (i.e. the data involved in the transaction, also known as transaction content) is written to the transaction log and buffer pool after the transaction is committed.

[0049] Page: also known as page, data page, physical page or physical page, page is the basic unit of database system storage. For example, in a certain database system, the size of the page is 8 kilobytes (KB), and there are 128 pages per megabyte in the database system.

[0050] Transaction log (redo log): in a relational database, it is used to record the data involved in the execution of the transaction. The transaction log is a set of files that can be directly stored on the disk, which is usually a sequential storage physical log. The transaction log is used to record the historical modification information (usually physical modification information) of the page. For example, a certain transaction log is used to record the historical modification information of the 10th byte to the 13th byte on a page. Using transaction log to record the historical modification information of the page is more efficient than regenerating a page, so as to ensure high write efficiency. The transaction log is usually stored in one or more log files (such as 2 log files), and the log file stores transaction records, each transaction record is also called a transaction entry, which is a set of change vectors (cv), also known as change carriers, each change carrier describes the change of a single data block in the database. For example: the salary value in an employee table is modified, which will generate a transaction record, which includes a change carrier that describes the change of the data block in the employee table. In a relational database, each database instance is provided with a corresponding transaction log, that is, the transaction log can represent a database instance.

[0051] After a data query instruction is processed through a transaction, the corresponding transaction data has been changed in the transaction log and the buffer pool, but has not yet been recorded in the storage medium (such as a disk). The page different from the recorded content in the buffer pool is called a dirty page. Based on the data in the buffer pool, the process of recording the dirty page to the storage medium is called flushing. If there is still part of the dirty page that has not been written to the storage medium, a failure occurs, such as a crash, and the data in the buffer pool will be lost. Therefore, the transaction log is mainly used for data recovery in a relational database, to ensure the consistency and integrity of the data in the relational database.

[0052] Metadata: also known as intermediate data or relay data, is data about data, and is usually information used to describe data properties. Metadata can be used to support functions such as indicating storage locations, historical data, resource searching, or file recording.

[0053] Sharding: is an independent data area (also known as a storage space) in a data storage node. The area can be physically isolated from other shards, or can be logically isolated.

[0054] Log sequence number (LSN): is an incremental integer number representing the total amount of bytes written to the transaction log by a transaction. Since it is always increasing, LSN can be used as a unique identifier for transaction data written at different times. LSN is mainly used for data recovery when a database fails. LSN can be saved in the transaction log or in the buffer pool.

[0055] Falling disk: refers to writing data to a storage medium. The storage medium is a non-volatile computer-readable storage medium, such as a disk, a read-only memory, or an optical disk.

[0056] Write ahead log (WAL) principle: also known as the log first principle, which means that when a transaction is committed, the transaction log is first written to the storage medium, and then the page data is modified. Here, the page data refers to the data pages and index pages in the buffer pool.

[0057] Snapshot: A fully usable copy of a specified set of data, including an image of the data at a certain point in time (the moment the copy begins). A snapshot can be a copy of the data it represents, or a replica of the data. Snapshots in this application's embodiments may refer to snapshots of the New Technology File System (NTFS) or the Fourth Extended File System (EXT4). Snapshots applied to databases are also called database snapshots.

[0058] The log data mentioned in the embodiments of this application are all transaction log data. Below is a brief introduction to the log data storage principle of relational databases: When the relational database is a distributed database, such as... Figure 1 As shown, the transaction log data is divided into multiple log shards, which are stored in shards on the data storage node. The data in each shard is called shard data. Typically, each shard data includes at least shard log data and may also include shard page data. The shard log data contains a portion of the log data, and the shard page data is generated based on the corresponding shard log data. For example, shard page data is obtained by the data storage node by converting the corresponding shard log data. It is worth noting that when a shard log data is converted into shard page data, that shard log data can be deleted or retained; this embodiment does not impose any limitations on this. Since shard data is generated based on log data, after obtaining multiple shard data, if log data needs to be recovered, it can be recovered based on the obtained shard data.

[0059] This application provides a data backtracking method applicable to distributed relational databases. For example, the distributed relational database can be built on an NTFS or EXT4 file system, or on other distributed file systems or distributed storage systems, such as Greenplum Database or TaurusDB. Greenplum DB is abbreviated as gpdb. Please refer to [link / reference]. Figure 2 , Figure 2is a schematic diagram of an application environment of a distributed database system (DDBS) involved in a data rollback method provided by an embodiment of the present application. The DDBS can be a server or a server cluster composed of multiple servers, and the DDBS includes a distributed database management system (DDBMS) and a DDB. In the distributed database system, an application program can transparently operate on the DDB through the DDBS, the data in the DDB is stored in different local databases, managed by one or more DDBMSs, run on different machines, supported by different operating systems, and connected together by different communication networks. Among them, the DDBS 10 includes a management node (also known as a database engine, a coordinator data node) 101 and multiple data nodes 102. For example, the DDBMS can be deployed on the management node 101, and the DDB can be deployed on the multiple data nodes 102.

[0060] The DDBS 10 can include one or more management nodes 101, which are used to manage corresponding data nodes 102 and implement operations of the application program on the data nodes 102, such as performing data addition, data deletion, data modification, or data query operations.

[0061] In an embodiment of the present application, the management node 101 can be a separate node, or a designated data node or an elected data node in the multiple data nodes 102, which can be a server or a server cluster composed of multiple servers. In one implementation, each data node can be a server or a server cluster composed of multiple servers; in another implementation, each data node represents a set minimum processing unit of the DDBS. For example, each data node can be a virtual machine or a container, which manages and / or stores an application instance or a database execution process. The distributed database can have one or more table spaces, which are logical divisions of the relational database, and a table space usually belongs to a database. A table space usually includes one or more relational tables.

[0062] Further, the management node 101 can also perform rollback of log data of a transaction log and corresponding shard data. As shown in Figure 3 Figure 3 is a schematic diagram of another application environment of a distributed database system involved in a data rollback method provided by an embodiment of the present application. Figure 3 ​Taking one management node 101 in the DDBS 10 as an example, it is assumed that the management node 101 comprises a management sub-node 1011, a backtracking sub-node 1012, and a virtual router 1013, a plurality of data nodes 102 comprising a plurality of data storage nodes including data storage nodes A to G. The plurality of data storage nodes generally occupy the memory space of the distributed database system.

[0063] The management sub-node 1011 is configured to receive a data processing request (such as a data read, write, add, delete, or modify request), and based on the received data processing request, perform data processing on a target data storage node through the virtual router 1013. The management sub-node 1011 supports Structured Query Language (SQL) functions. The functions of the management sub-node 1011 can be implemented by a database instance, which is the subject of data writing. The backtracking sub-node 1012 is a scheduler of a backtracking process, configured to send each request (or instruction) in the backtracking process to each data node in the DDBS through the virtual router 1013 to backtrack the log data and the shard data. The request can include a backtracking instruction or a snapshot request involved in subsequent embodiments. The virtual router 1013 is configured to route the request to achieve distribution of the request for execution of an action corresponding to the request, such as reading and writing data, by the data node receiving the request. The functions of the management sub-node 1011, the backtracking sub-node 1012, and the virtual router 1013 can be implemented by an independent process respectively, or by the same process, which is a process in the database instance.

[0064] The data storage nodes can be divided into log storage nodes and shard storage nodes due to the different types of data stored therein. The log storage nodes are configured to store log data, each log data having one or more copies, referred to as log data copies, and the content of one log data being stored through its copies. Different copies of the same log data are distributed in different log storage nodes. The shard storage nodes are configured to store shard data, each shard data having one or more copies, and the content of one shard data being stored through its copies. Different copies of the same shard data are distributed in different shards of different shard storage nodes. Figure 3It is assumed that the data storage nodes A to C are log storage nodes, and the data storage nodes D to G are shard storage nodes. The log data has three copies, namely log data copies 1 to 3, which are respectively stored in the data storage nodes A to C. The three shard data corresponding to the aforementioned log copies are shard data 1 to 3. Each shard data has three copies, namely copies 1 to 3, which are respectively stored in three data storage nodes in the data storage nodes D to F. For example, the copies 1 to 3 of the shard data 1 are respectively stored in the data storage nodes D, E and G; the copies 1 to 3 of the shard data 2 are respectively stored in the data storage nodes D, E and G; and the copies 1 to 3 of the shard data 3 are respectively stored in the data storage nodes D, F and G.

[0065] It should be noted that the data rollback method provided by the embodiments of the present application adopts an append only mode for writing the transaction log or the page, which is also called a random read and append write mode. The mode supports only new data writing and does not modify the written data by overwriting.

[0066] In an optional manner, the file system of the distributed relational database provided by the embodiments of the present application itself supports the append only mode. In the file system in the append only mode, only files can be appended but not rewritten, and the generated file is called an append only file (AOF).

[0067] In another optional manner, the file system of the distributed relational database provided by the embodiments of the present application itself does not set the append only mode. However, as long as the write mode of the transaction log or the page adopts the append only mode, the data rollback method provided by the embodiments of the present application can be used for data rollback, regardless of whether the distributed relational database is constructed based on the NTFS or EXT4 file system or based on other distributed file systems or distributed storage systems.

[0068] For the convenience of the reader to understand, the principle of the data rollback method provided by the embodiments of the present application is introduced as follows. In a distributed database, based on the log-first principle, after each transaction log is generated, the log data of the transaction log is first written to the corresponding data storage node to realize the landing of the log data (i.e., storing a specified number of copies of the log data). After the log data of the current transaction log (i.e., this transaction log) is landed, it is divided into multiple shard data, and then written to the corresponding data storage node to realize the landing of the shard data (i.e., storing a specified number of copies of each shard data). The shard data includes at least one of shard log data and shard page data. The shard page data is generated based on the corresponding shard log data, for example, the shard page data is obtained by converting the corresponding shard log data. In a traditional distributed relational database, metadata is not set for the log data and the shard data, and the embodiments of the present application configure metadata for the log data and the shard data to realize data rollback.

[0069] The metadata of the log data and the metadata of the shard data in the embodiments of the present application are introduced as follows: the data corresponding to the transaction log includes the log data and the metadata of the log data. The log data is the data involved in the execution process of the transaction, also known as transaction content; the metadata is used to describe the log data. Through the metadata, the log data can be located, that is, the storage location of the log data is determined.

[0070] Due to the data backtracking method provided in the embodiments of the present application, the writing mode of the transaction log or the page adopts an append-only mode, and in the file system under the mode, the size of the file (i.e. the amount of data that can be written in the file) is set with an upper limit. If the data to be written is large (i.e. the size of the data is larger than the upper limit), it needs to be written in multiple files (i.e. the preset minimum storage unit). Therefore, after the data is written to disk, the written data is stored in at least one file. For example, a log data is stored in at least one file (usually at least two files), and a page data is stored in at least one file. When the written data needs to be stored in multiple files, the multiple files need to have a certain order. When data is backtracked, the multiple files can be sequentially linked, that is, the multiple files have a prior relationship, and through the prior relationship, the multiple files can be connected. Therefore, in the embodiments of the present application, the metadata of the log data can include: data writing information of each file in the at least one file, the at least one file being used to store the log data. That is, each file includes two parts: one part is used to store log data, and the other part is used to store data writing information. Among them, a specified position (also can be called a specified space) of a fixed size can be set in each file to store the data writing information. For example, the specified position can be the file header or the file tail of the file. The data writing information is used to describe the writing situation of the log data in the file. For example, the data writing information is used to record the position of the corresponding file in the at least one file to which it belongs, and the end position of the written data (i.e. the written log data) in the corresponding file. Through such metadata, the positions of the files can be effectively identified, so that when data is backtracked, the files can be sequentially and continuously acquired, the continuity of the files is ensured, and effective log data is obtained through backtracking.

[0071] Since in the file system in which the writing mode of the transaction log or the page is in the append-only mode, the file can only be written and the data amount of the file is incremental. Therefore, the end position of the written data in the file can be identified by the data amount of the written file. Alternatively, for any file, the total amount of the currently written data (i.e. the sum of the data amounts written at all writing time points before the current time point of the file) can be used to identify the end position of the written data in the file after each time of writing data. For example, assuming that for the file F1, 100 bytes of data written at 3 time points are recorded, and correspondingly, the management node determines the data amount written at the corresponding time point and records the data amount each time of writing data. Assuming that the management node writes 3 indication information at the specified position at 3 time points, and the 3 indication information respectively indicates the data amount of 100 bytes, 200 bytes and 300 bytes. For example, the indication information is the data amount value, such as 100, 200 and 300. In this way, each time the end position of the written data is queried, the indication information of the recently written data amount is directly queried to quickly determine the end position of the written data. For example, the recently written indication information is 300 obtained by the query, and the end position of the written data is determined as the position of writing 300 bytes of data.

[0072] Currently, the log data of each transaction log in the transaction log includes one or more transaction records, and each transaction record carries an LSN. As shown in Figure 4 Figure 4 is a structure diagram of a transaction record provided by an embodiment of the present application. The transaction record includes a log type bit, a log length bit, an LSN bit, a data bit and a check bit. The log type bit is used to record the modification mode (also referred to as the modification type) of the transaction log to the page, for example, the content of the log type bit is used to indicate that 2 bytes are modified, an empty page is created or a file name is modified, etc. The log length bit is used to record the length of the transaction record, for example, the length is 100 bytes. The LSN bit is used to record the LSN of the transaction record. The data bit is used to record the actual data (or actual content) in the transaction record. The check bit is used to record a check value, and the check value is used to check the accuracy of the data carried by the data bit. The lengths of the log type bit, the log length bit, the LSN bit and the check bit are fixed. For example, the length of the log type bit is 4 bytes, the length of the log length bit is 8 bytes, the length of the LSN bit is 8 bytes, and the length of the check bit is 8 bytes.

[0073] ​As mentioned above, LSN represents the total amount of bytes of transaction written to transaction log. Obviously, LSN is an indication of typical data amount of transaction log. Since log data of a file is written continuously, as the number of transaction records written increases, LSN also gradually increases. Therefore, the maximum LSN carried by the written data in a file (i.e. the LSN of the most recently written transaction record) can reflect the end position of the written data. In the embodiments of the present application, for any file of the at least one file corresponding to the log data, the data write information of the any file can comprise: the maximum LSN carried by the written data in the any file, so as to identify the end position of the written data.

[0074] For example, for file F2, it records 3 times of written data. Correspondingly, at each time of writing data, by reading the LSN bit in the transaction record in the data written at the corresponding time (the structure of the transaction record can refer to the aforementioned Figure 4 ), the maximum LSN of the written data is determined, and the maximum LSN of the written data is recorded in the metadata, so as to identify the end position of the written data. Assuming that the maximum LSN of the 3 times of written data is 0xFFFFFFFFF000, 0xFFFFFFFFF00F and 0xFFFFFFFFF0FF respectively. Then the LSN recorded in the file F2 by the management node finally comprises: 0xFFFFFFFFF000, 0xFFFFFFFFF00F and 0xFFFFFFFFF0FF. Among them, the maximum LSN: 0xFFFFFFFFF0FF identifies the end position of the most recently written data.

[0075] Optionally, the data write information of any file corresponding to the log data further comprises: file position identifier of the any file. The file position identifier is used to identify the position of the any file in the at least one file to which it belongs. Wherein, the file position identifier can be composed of one or more characters. For example, the character can be binary character, such as 0 or 1, or decimal character.

[0076] In an optional manner, since the identification of some files has the function of identifying its position, the file position identifier can be the identification of the file. For example, the log data is stored in 3 files, the identification of the three files is assigned according to the arrangement order, which is 00, 01 and 10 respectively, and the file position identifier in file 00 is 00.

[0077] In another alternative, the file position identifier can include an identifier of a previous file and / or an identifier of a next file of the any file. It is worth mentioning that when the any file is the first file of the at least one file, the previous file of the any file is empty; when the any file is the last file of the at least one file, the next file of the any file is empty; when the log data is stored in only one file, i.e., the at least one file is actually only one file, the data write information in the file includes an identifier of a previous file and an identifier of a next file of the file, both of which are empty. Still taking the example that the log data is stored in the aforementioned three files, the file position identifier in the file 00 is 01 (i.e., only including the identifier of the next file); the file position identifier in the file 01 is 00+10 (i.e., including the identifier of the previous file and the identifier of the next file); and the file position identifier in the file 10 is 01 (i.e., only including the identifier of the previous file).

[0078] In an embodiment of the present application, the data corresponding to each shard includes shard data and metadata of the shard data. The metadata is used to describe the shard data. The shard data can be located by using the metadata, i.e., the storage location of the shard data is determined.

[0079] Since the shard log data is obtained by dividing the log data, each shard log data is actually a part of the log data and has the same attributes as the log data. Referring to the aforementioned log data, after the shard log data is written to a disk, one (or one copy) of the shard log data is stored in at least one file. In addition, the data storage node can convert part of the shard log data in a shard into shard page data, and the shard page data is stored in at least one file. It is worth mentioning that the files in which the shard log data and the shard page data of the same shard are stored can be the same or different. For example, in the same shard, the shard log data is stored in files 1 to 3, and the shard page data is stored in files 4 to 6; for another example, the shard log data is stored in files 1 to 6, and the shard page data is also stored in files 1 to 6, wherein each file stores part of the shard log data and part of the shard page data. The storage manner of the shard log data and the shard page data in the files is set according to the storage rule of the database system, and the embodiment of the present application does not limit this.

[0080] In the embodiment of the present application, the metadata of the sharded data can include: data write information of each file in the at least one file used to store the log data. That is, each file used to store the sharded data includes two parts: one part is used to store the sharded data, and the other part is used to store the data write information. A specified position of a fixed size can be set in each file to store the data write information. For example, the specified position can be the file header or the file tail of the file. The data write information is used to describe the write condition of the sharded data in the file, for example, the data write information is used to record the position of the corresponding file in the at least one file to which the corresponding file belongs, and the end position of the write data (i.e., the written sharded data) in the corresponding file. Through such metadata, the positions of the files can be effectively identified, so that when data is traced back, the files can be sequentially and continuously acquired, the continuity of the files is ensured, and effective sharded data is recovered.

[0081] Similarly to the file used to store the log data, in the embodiment of the present application, for any file in the at least one file corresponding to the sharded data, the data write information of the any file includes: the maximum LSN carried in the write data in the any file, so as to identify the end position of the write data.

[0082] Optionally, the data write information of any file corresponding to the sharded data further includes: a file position identifier of the any file. The file position identifier is used to identify the position of the any file in the at least one file to which the any file belongs. The structure of the file position identifier can refer to the structure of the file position identifier of the file corresponding to the log data. The structure of the file used to store the sharded data can be the same as or similar to the structure of the file used to store the log data, and thus the structure of the file of the sharded data is not described herein.

[0083] The embodiment of the present application provides a data tracing back method, which is applied to an application environment as shown in Figure 2 or Figure 3 The data tracing back method can be executed by the database system, for example, by the management node, and further, by the tracing back sub-node in the management node. Figure 5 FIG. 1 is a flow diagram of a data tracing back method provided by the embodiment of the present application, as shown in Figure 5 The data tracing back method is executed by the management node, and subsequent embodiments take the data tracing back method of the log data and the sharded data of one database as an example for description, which includes the following steps.

[0084] S501, the management node stores the log data.

[0085] When the management node detects the data processing request sent by the application, the management node controls the corresponding data node to execute the operation requested by the data processing request through a transaction. For example, the management node can initiate multiple distributed transactions based on the data processing request, generate one or more distributed plans when each distributed transaction is executed, and instruct the corresponding data node to execute the generated distributed plan, so as to implement the operation requested by the data processing request.

[0086] In the foregoing transaction execution process, corresponding transaction data is generated. The management node writes the transaction data into the transaction log to implement storage of the log data, that is, landing of the log data.

[0087] S502, the management node stores the sharding data corresponding to the log data.

[0088] Optionally, the database system stores sharding relationship data, which is used to record the correspondence between the sub-data of the log data and the shards. The sub-data is a pre-divided data allocation unit and is the smallest storage unit supported by the database and cannot be modified. One (or one copy) of the sub-data can include one or more transaction records. The sharding relationship data can be stored in the form of a table, and thus is also referred to as a sharding list. The sharding list can update the correspondence between the sub-data of the log data and the shards in a full update manner, that is, a new sharding list is used to replace the original sharding list each time the correspondence needs to be updated. The sharding list can also update the correspondence between the sub-data of the log data and the shards in an incremental update manner, that is, after the sharding list is saved for the first time, if the correspondence needs to be updated, only the data different from the data recorded in the original sharding list is updated, that is, the incremental data is updated.

[0089] Optionally, the sharding relationship data can include first correspondence data and second correspondence data. The first correspondence data is used to record the correspondence between the sub-data of the log and the database object, and the second correspondence data is used to record the correspondence between the database object and the shard. The database object refers to an object to which a data processing process in a relational database is directed. The database object can be a table space (such as a relational table), a page, an index, or another object related to database classification (also referred to as attribution).

[0090] In an example, the first correspondence data can record the correspondence between the identification information of the sub-data and the identification information of the page, and the second correspondence data can record the correspondence between the identification information of the page and the identification information of the shard. The correspondence between the identification information of the sub-data and the identification information of the page is usually a many-to-one correspondence, and the correspondence between the identification information of the page and the identification information of the shard is usually a one-to-one or many-to-one relationship. For example, one shard can correspond to 100 pages. The identification information of the page is used to uniquely identify a page, and the identification information can include one or more characters. For example, the characters can be binary characters such as 0 or 1, or decimal characters. It should be noted that one relational database can store one or more relational tables, and each relational table includes multiple pages. In order to uniquely identify a page, when the relational database stores only one relational table, the identification information of the page can include only the identity document (ID) of the page; when the relational database stores multiple relational tables, the identification information of the page can include the ID of the relational table and the ID of the page.

[0091] The identification information of the shard is used to uniquely identify a shard, and the identification information can include one or more characters. For example, the characters can be binary characters such as 0 or 1, or decimal characters. It should be noted that when the distributed relational database includes multiple data storage nodes, each data storage node includes multiple shards. In order to uniquely identify a shard, in an optional manner, all shards can be assigned different shard IDs as the identification information of the corresponding shard; in another optional manner, the shard IDs on different data storage nodes can be the same, and the identification information of the shard can include the ID of the data storage node and the shard ID.

[0092] Each database object generates log data when being operated. The management node can perform shard processing on the log data based on the shard relationship data. For example, for each sub-data of the obtained log data, the management node can query the first correspondence data to obtain the database object corresponding to the sub-data; then, the management node queries the second correspondence data using the database object corresponding to the sub-data to obtain the corresponding shard (i.e., determining the shard corresponding to each sub-data, and this process can be regarded as mapping each sub-data to the corresponding shard). Finally, the management node determines the sub-data corresponding to the same shard as a shard log data, and then writes each shard log data into the corresponding shard, thereby completing the storage of the shard log data. After one or more shard log data are stored, the shard log data can be converted into shard page data, and finally the shard data stored in one shard can include shard log data and / or shard page data.

[0093] It is worth noting that the aforementioned sharding relationship data can also only include the aforementioned second correspondence relationship data, each sub-data carries the identification information of its corresponding database object (also known as the owned database object) when generated. For each sub-data of the obtained log data, the management node can query the second correspondence relationship data using the identification information of the database object carried by the sub-data to obtain the corresponding shard. There is no need to query two correspondence relationships. For example, assuming that the database object is a page, the second correspondence relationship data can be as shown in Table 1. The pages with page IDs 00 to 04 are one-to-one corresponding to the shards with shard IDs 0 to 3, respectively.

[0094] Table 1

[0095]

[0096]

[0097] Assuming that it is determined through querying the first correspondence relationship data or extracting the page ID carried in the sub-data that the current obtained log data includes: sub-data A belonging to page 00 and sub-data B belonging to page 01, querying the above table 1 can write sub-data A as shard log data to shard 0; and write sub-data B as shard log data to shard 1. Thus, the storage of the shard log data included in the log data is realized.

[0098] Optionally, a plurality of page index data corresponding to a plurality of shards are stored in the database system, and the page index data are used to record the correspondence relationship between the shard log data and the shard page data in the corresponding shard. Based on the page index data of each shard, it can be determined which shard log data in the shard is converted into shard page data.

[0099] In the embodiment of the present application, the database supports a data rollback function, which can be pre-set in the database or started after receiving a user-triggered database function start instruction. The data rollback function supports the collection of metadata of log data and metadata of shard data. After the data rollback function is started, the management node can perform data rollback based on the records of the collected metadata. In general, the data rollback function is started at the initialization, migration or other specified moment of the database.

[0100] S503, the management node obtains the metadata of the log data corresponding to n first collection moments, n being a positive integer.

[0101] Optionally, the management node obtains the metadata corresponding to the n first collection time points obtained by performing the n times of metadata collection processing on the log data. For example, the management node can perform one or more times of metadata collection on the log data after the database system is deployed, so as to realize backup of the metadata of the log data, and thus data can be traced back in the subsequent process.

[0102] The first collection time point can be a collection start time point of the metadata of the log data, or a collection end time point of the metadata of the log data, or a first set time point. Since the data amount of the metadata of the log data is small, the collection start time point is approximately equal to the collection end time point, that is, the time difference between the two time points can be ignored.

[0103] The metadata corresponding to each first collection time point includes data write information of each file in at least one file used for storing the log data. For example, the metadata corresponding to the first collection time point 9:00 includes data write information of files 100 to 120 used for storing the log data, and the metadata corresponding to the first collection time point 9:30 includes data write information of files 110 to 130 used for storing the log data. It can be known that in the time period from 9:00 to 9:30, files 100 to 109 are deleted and files 121 to 130 are added in the database system.

[0104] In an optional manner, the management node can periodically perform metadata collection on the log data. The collection period can be pre-set, and the smaller the interval is, the more the first collection time points that can be selected for tracing back. For example, the time intervals of adjacent collection periods are the same.

[0105] In another optional manner, the management node can collect the metadata of the log data in the database system after receiving a data collection instruction.

[0106] Optionally, the management node can obtain the metadata of the log data by using snapshot, that is, the metadata collection processing is snapshot processing. In this way, the speed of obtaining the metadata can be effectively accelerated, and the collection efficiency can be improved.

[0107] S504, the management node obtains the metadata of the sharded data corresponding to the m second collection time points, m being a positive integer.

[0108] Optionally, the management node obtains the metadata corresponding to the m second collection time points obtained by performing m times of metadata collection processing on the sharded data.

[0109] In the embodiments of the present application, it is assumed that the sharded data in S502 is obtained by processing the log data in S501. Similarly, the management node can collect the metadata of the sharded data (i.e., the data in each shard) one or more times after the deployment of the database system, to implement the backup of the metadata of the sharded data, so that the data can be traced back in the subsequent process.

[0110] The second collection time can be the collection start time of the metadata of the plurality of sharded data, or the collection end time of the metadata of the plurality of sharded data, or a second set time. When the number of sharded data corresponding to the log data is large, the time difference between the collection start time and the collection end time is large, and the collection end time is usually taken as the second collection time.

[0111] Each second collection time corresponds to the metadata of a plurality of sharded data of the database system, and usually corresponds to the metadata of all sharded data of the database system at the second collection time, so that if it is necessary to trace back to the second collection time, the data obtained by tracing back is all the sharded data in the database system at the second collection time.

[0112] Since there are a plurality of sharded data, each second collection time corresponds to a group of metadata, which can also be regarded as a metadata sequence. For the metadata of each sharded data, it includes data write information of each file in at least one file used to store the sharded data. For example, it is assumed that the first sharded data is any sharded data of the log data in the database system, and for the first sharded data, the metadata corresponding to the second collection time 9:00 includes data write information of files 110 to 115 used to store the first sharded data, and the metadata corresponding to the second collection time 9:30 includes data write information of files 111 to 120 used to store the first sharded data, so it can be known that in the period from 9:00 to 9:30, the database system deletes file 100 and adds files 116 to 120 in the first shard where the first sharded data is located. The collection principle of the metadata of other sharded data of the log data in the database system can refer to the collection principle of the metadata of the first sharded data, and the embodiments of the present application do not repeat the description.

[0113] In an optional manner, the management node can periodically collect the metadata of the sharded data. The collection period can be pre-set, and the smaller the interval, the more the second collection times that can be selected for tracing back. For example, the time intervals of adjacent collection periods are the same.

[0114] In another optional manner, the management node can collect the metadata of the sharded data in the database system after receiving a data collection instruction.

[0115] In the embodiment of the present application, when collecting a plurality of pieces of shard data at one time, the management node can sequentially (also referred to as serially) acquire the metadata of each piece of shard data in the plurality of pieces of shard data, or can acquire the metadata of each piece of shard data in the plurality of pieces of shard data in parallel. Optionally, the management node can acquire the metadata of the shard data by using a snapshot, that is, the aforementioned metadata collection processing is a snapshot processing. In this way, the speed of acquiring the metadata can be effectively accelerated, and the collection efficiency can be improved.

[0116] It should be noted that, in the distributed database, since the data rollback needs to ensure the consistency of the log data and the shard data obtained by the rollback, the log data and the shard data also need to have a certain corresponding relationship in time during the metadata collection phase. The embodiment of the present application is described by taking the following two optional implementation modes as examples:

[0117] In the first optional implementation mode, the n first collection time points and the m second collection time points have a one-to-many corresponding relationship, that is, each first collection time point corresponds to at least two second collection time points.

[0118] For example, it is assumed that the corresponding relationship between the first collection time point and the second collection time point is a 1-to-2 relationship. If the management node periodically collects the metadata of the log data and the metadata of the shard data, then two times of collection of the metadata of the shard data are performed within a collection period corresponding to one time of collection of the metadata of the log data. Optionally, the time length of the collection period of the metadata of the log data is twice the time length of the collection period of the metadata of the shard data. If the management node collects the metadata of the log data and the metadata of the shard data after receiving a data collection instruction, then one time of collection of the metadata of the log data and two times of collection of the metadata of the shard data are performed after receiving the data collection instruction, and the interval time length of the two times of second collection of the metadata of the shard data can be a pre-set time length.

[0119] In the second optional implementation mode, m is equal to n, the n first collection time points and the m second collection time points correspond to each other, and the time difference between the corresponding first collection time point and the second collection time point is less than a difference threshold. For example, the first collection time point x1 corresponds to the second collection time point x2, the first collection time point x1 is a time point after the second collection time point x2, and the time difference between the two is less than a difference threshold (if the metadata of the log data and the metadata of the shard data are both collected by using a periodic collection mode, the difference threshold is less than a collection period), or the first collection time point x1 is equal to the second collection time point x2. By setting the first collection time point and the second collection time point to be close to or the same, the metadata collection process can be simplified, and subsequent data rollback can be facilitated.

[0120] S505, the management node acquires the first metadata corresponding to the target historical time in the metadata of the log data.

[0121] In order to improve the disaster recovery of the database, it is necessary to backtrack the data in some scenarios. For example, in a certain period of time, the database has error data writing, and it is necessary to backtrack the data of the database to the period before.

[0122] The management node can perform the backtracking operation after receiving the backtracking instruction indicating the target historical moment. In an optional manner, the management node can present n first collection moments through a user interface, select a first collection moment as the target historical moment from the n first collection moments by a user, and generate the backtracking instruction when detecting the selection operation. In another optional manner, the management node can present a time axis through a user interface, indicate a target moment by a user in the time axis (for example, drag or click on the time axis to indicate the target moment), and take the first collection moment closest to the target moment in the n collection moments as the target historical moment, and generate the backtracking instruction when detecting the selection operation.

[0123] In the embodiment of the present application, the management node can first backup all the write data in the database after receiving the backtracking instruction, to prevent the case of data backtracking failure in extreme cases. If the data backtracking failure case occurs in the subsequent process, the database can be rolled back to the correct state based on the backup write data.

[0124] It should be noted that after receiving the backtracking instruction, the database system will enter the backtracking state, and the database system entering the backtracking state will perform a preprocessing operation to ensure the effective execution of the subsequent data backtracking action. For example, the preprocessing operation includes one or more of the following: suspending database use, closing user connection (the user connection refers to the connection between the database and the application), discarding uncommitted write data, or cleaning up data in memory, etc.

[0125] Referring to the foregoing S503, the management node obtains the metadata corresponding to the n first collection moments, and in S505, after determining the target historical moment, the management node can obtain the first metadata corresponding to the target historical moment from the metadata corresponding to the n first collection moments. Referring to the example in S503, if the target historical moment is 9:00, the first metadata corresponding to the target historical moment is the metadata of files 100 to 120.

[0126] S506, the management node obtains the second metadata corresponding to the target historical moment from the metadata of each shard data in the plurality of shard data, and the plurality of shard data is obtained by performing the shard processing on the log data.

[0127] Referring to S504 and S505, the management node has obtained the metadata corresponding to the m second collection time, and the management node has determined the target historical time, so for each piece of shard data, the management node can obtain the second metadata corresponding to the target historical time in the metadata corresponding to the m second collection time. Referring to the foregoing S504, since the log data and the shard data also have a certain correspondence in time in the metadata collection stage, the management node can obtain the second metadata corresponding to the target historical time in the metadata of each piece of shard data in the plurality of shard data based on the correspondence. The embodiments of the present application are described by taking the following two optional implementation manners as examples:

[0128] Corresponding to the first optional implementation manner of the foregoing S504, in the first optional implementation manner, since each first collection time corresponds to at least two second collection times, the management node can select a target collection time from the at least two second collection times corresponding to the target historical time, and obtain the metadata corresponding to the target collection time as the second metadata corresponding to the target historical time. For example, the target collection time is a randomly selected collection time from the at least two second collection times, or a collection time farthest from the target historical time.

[0129] Corresponding to the second optional implementation manner of the foregoing S504, in the second optional implementation manner, since each first collection time corresponds to one second collection time, the management node can obtain the metadata corresponding to the second collection time corresponding to the target historical time as the second metadata corresponding to the target historical time.

[0130] For example, when the target historical time is a first collection time x in the n first collection times, the second collection time corresponding to the target historical time is the second collection time corresponding to the first collection time x; when the target historical time is a first collection time y closest to the target time indicated by the user in the n collection times, the second collection time corresponding to the target historical time is the second collection time corresponding to the first collection time y.

[0131] Referring to the example in S504, if the target historical time is 9:00, the second metadata corresponding to the target historical time is the metadata of files 110 to 115.

[0132] S507, the management node processes the log data according to the first metadata to obtain log data of the target historical time.

[0133] As described in S503, each time the collection processing collects a metadata reflecting the state of the log data of the database at the collection time, the management node can obtain the log data at the target historical time by rolling back the log data according to the data location indicated by the first metadata. Referring to the example in S506, the first metadata corresponding to the target historical time includes the data write information of the files 100-120 used to store the log data, and the data in the files 100-120 can be recovered based on the first metadata. The recovered log data is the log data at the target historical time.

[0134] In an optional implementation, the management node stores backup data of the database system in the local storage space of the database or the cloud storage space, and the backup data includes log data at different time periods in the database system. The management node can directly extract the data at the corresponding location in the backup data based on the first metadata, and thus obtain the log data at the target historical time.

[0135] In another optional implementation, if the current log data stored by the management node when receiving the rollback instruction includes the data in the data location indicated by the first metadata, the management node can delete the data in the current log data except the data in the data location indicated by the first metadata, and thus obtain the log data at the target historical time. For example, the files used to store the log data by the management node when receiving the rollback instruction are files 100-150, and the files indicated by the first metadata are files 120-140. The management node can delete files 100-119 and 141-150 in the files 100-150, and thus obtain the log data at the target historical time.

[0136] S508, the management node processes the plurality of shard data according to the second metadata, and obtains the plurality of shard data at the target historical time.

[0137] For example, the management node can roll back the plurality of shard data according to the data location indicated by the second metadata, and thus obtain the plurality of shard data at the target historical time. As described in S504, for each shard data, each time the collection processing collects a metadata reflecting the state of one or more shard data of the database at the collection time. The second metadata corresponding to a second collection time is a set of a plurality of metadata corresponding to the plurality of shard data. The management node can roll back the shard data according to the data location indicated by the second metadata corresponding to the shard data, and thus obtain the shard data at the target historical time. Referring to the example in S506, the second metadata corresponding to the first shard data at the target historical time includes the data write information of the files 110-115 used to store the first shard data. The data in the files 100-120 can be recovered based on the second metadata. The recovered data is the first shard data at the target historical time.

[0138] Referring to the two optional implementations of S507, the management node stores backup data of the database system in the database local storage space or the cloud storage space, and the backup data includes multiple shard data of different periods in the database system. The management node directly extracts data at the corresponding position in the backup data based on each second metadata, so as to obtain one shard data of the target historical moment. Alternatively, if the management node stores a certain shard data in the current shard data when receiving the rollback instruction, and the certain shard data includes data in the data position indicated by the corresponding second metadata. The management node can delete data in the certain shard data except the data position indicated by the second metadata, so as to obtain one shard data of the target historical moment.

[0139] In summary, in the embodiments of the present application, the first metadata corresponding to the target historical moment of the log data is used to rollback the log data, and the second metadata corresponding to the target historical moment of the shard data is used to rollback the shard data, without performing inverse operations of logical record reading and updating operations, so that the data rollback efficiency is higher, and the data rollback delay is effectively reduced.

[0140] In the foregoing S503 to S504, since the collection of the metadata of the log data and the metadata of the shard data is performed respectively, for example, two processes are used to perform the collection, therefore, the corresponding relationship in time between the log data and the shard data needs to be recorded, so as to facilitate subsequent data rollback, and to realize effective rollback of the associated log data and shard data (that is, the log data and the shard data corresponding to the target historical moment).

[0141] In actual implementation, the metadata of the log data and the metadata of the shard data are collected in association, so that the collected metadata can be stored in association, without recording the foregoing corresponding relationship in time, and accordingly, the process of the management node obtaining the first metadata and the second metadata corresponding to the target historical moment is simplified, thereby reducing the complexity of data rollback. Figure 6 is a flow diagram of another data rollback method provided by the embodiments of the present application, as shown in Figure 6 It is assumed that the data rollback method is performed by the management node, and subsequent embodiments take the data rollback method of log data and shard data of one database as an example for description, and the method includes:

[0142] S601, the management node stores log data.

[0143] The process of S601 can refer to the foregoing S501.

[0144] S602, the management node stores shard data corresponding to the log data.

[0145] The process of S602 can refer to the foregoing S502.

[0146] S603, the management node acquires metadata corresponding to k collection time points, the metadata corresponding to each collection time point of the k collection time points including metadata of log data and metadata of a plurality of piece data. K is a positive integer.

[0147] The metadata corresponding to each collection time point is actually a set of metadata. For example, the management node can periodically collect metadata of the database or collect metadata of the database after receiving a data collection instruction to obtain metadata corresponding to k collection time points. The collection time point can be a collection start time point, a collection end time point, or a time point between the collection start time point and the collection end time point (i.e., a time point in the metadata collection period), which is used to identify the state of each metadata in the database in the metadata collection period. It should be noted that if the collection period is long, the state of each metadata in the database in the metadata collection period can be accurately identified by using the collection end time point as the aforementioned collection time point, because the data may change from the collection start time point to the collection end time point.

[0148] It should be noted that when the aforementioned collection time point is the collection end time point of a set of metadata. Because when collecting a set of metadata, if the metadata of the piece data is too much, the time consumed for collecting the second metadata is too long, if the first metadata is collected first and then the second metadata is collected, it is easy to have the case that the log data generates incremental data in the period of collecting the second metadata, and the case that the data position indicated by the first metadata is inconsistent with the actual data position of the log data at the collection end time point. Therefore, in the collection process of each set of metadata, the metadata of the piece data is usually collected first, and then the metadata of the log data is collected, so as to reduce the inconsistency between the data position indicated by the first metadata and the actual data position of the log data at the collection end time point caused by the incremental log data.

[0149] S604, the management node acquires metadata corresponding to a target historical time point, the metadata corresponding to the target historical time point including first metadata and a plurality of second metadata.

[0150] The management node can perform the rollback operation after receiving the rollback instruction indicating the target historical moment. In an optional manner, the management node can present the k collection moments through a user interface, select one collection moment as the target historical moment by a user from the k collection moments, and generate the rollback instruction upon detecting the selection operation. In another optional manner, the management node can present a timeline through a user interface, indicate the target moment by a user on the timeline (for example, drag or click on the timeline to indicate the target moment), and determine the collection moment closest to the target moment from the k collection moments as the target historical moment, and generate the rollback instruction upon detecting the selection operation.

[0151] Referring to S603, the management node has obtained the metadata corresponding to the k collection moments, and the management node has determined the target historical moment, the management node can filter the metadata corresponding to the target historical moment from the metadata corresponding to the k collection moments, thereby determining the first metadata corresponding to the target historical moment and the plurality of second metadata.

[0152] S605, the management node processes the log data according to the first metadata to obtain the log data of the target historical moment.

[0153] The process of S605 can refer to the foregoing S507.

[0154] S606, the management node processes the plurality of shard data according to the second metadata to obtain the plurality of shard data of the target historical moment.

[0155] The process of S606 can refer to the foregoing S508.

[0156] In summary, in the embodiments of the present application, the first metadata corresponding to the target historical moment of the log data is used to rollback the log data, and the second metadata corresponding to the target historical moment of the shard data is used to rollback the shard data, without performing the inverse operation of the logical record reading and updating operation, the data rollback efficiency is higher, and the data rollback delay is effectively reduced.

[0157] As shown in Figure 1 Each shard data is determined based on the log data. In a distributed relational database, consistency (also referred to as business consistency or logical consistency) between the plurality of shard data and the log data at the same moment needs to be ensured. However, due to the inconsistency of the foregoing collection moments or other reasons, inconsistency between the log data of the target historical moment and the plurality of shard data of the target historical moment can occur. In order to facilitate the reader to understand, the following will be described in combination with Figure 7 The inconsistency between the log data of the target historical moment and the plurality of shard data of the target historical moment can occur. Figure 7 is a timeline diagram of a data rollback method provided by an embodiment of the present application. Figure 7Suppose the metadata of the log data and the metadata of the shard data are collected in association, and record the metadata acquisition process corresponding to the collection time, then the management node starts to collect the metadata of the log data and the metadata of the shard data when receiving the backtracking instruction or reaching the collection period. Figure 7 In the metadata collection by snapshot, the collection time is the end time of the metadata collection of the plurality of shard data. Figure 7 In the metadata collection by snapshot, the snapshot obtained by acquiring the metadata of the log data is the first snapshot, the snapshot obtained by acquiring the metadata of each shard data is the second snapshot, the plurality of second snapshots correspond to the metadata of the plurality of shard data one by one, and the collection period of the metadata in the database system at one time includes the metadata collection period L1 of the log data and the metadata collection period L2 of the shard data. Figure 7 In the metadata collection by snapshot, the start time of the collection period is the acquisition time of the second snapshot, and the end time of the collection period is the acquisition time of the first snapshot. The first collection time and the second collection time can be set as the end time of the collection period. If a log data corresponds to a plurality of shard data, the actual collection end time is far apart from the collection start time, and in the metadata collection period L2, the database system can generate incremental log data (i.e. newly written log data), and the records of the metadata of the incremental log data by the first snapshot and the second snapshot are not necessarily consistent, so that the log data and the shard data are inconsistent.

[0158] Therefore, after the foregoing S508 or S606, the management node can also update the plurality of shard data at the target historical time based on the log data at the target historical time, so that the plurality of shard data after the update is consistent with the log data.

[0159] In an optional implementation, the management node can detect whether the data end position of the log data at the target historical time is consistent with the data end position of the plurality of shard data, when detecting that the data end position of the log data at the target historical time is inconsistent with the data end position of the plurality of shard data, update the plurality of shard data at the target historical time based on the log data at the target historical time, so that the plurality of shard data after the update is consistent with the log data; and when detecting that the data end position of the log data at the target historical time is consistent with the data end position of the plurality of shard data, do not perform any action.

[0160] In another optional implementation, since the probability that the log data at the target historical time is inconsistent with the shard data at the target historical time is high, the management node can directly perform consistency processing. That is, update the plurality of shard data at the target historical time based on the log data at the target historical time, so that the plurality of shard data after the update is consistent with the log data.

[0161] The consistency processing process includes, for example, updating the plurality of shard data of the target historical moment based on the shard relationship data and the log data of the target historical moment. The plurality of shard data after the update is consistent with the log data. Referring to S502, the shard relationship data is used to record the correspondence between the sub-data in the log data and the shard.

[0162] The management node can obtain the shard relationship data in multiple ways. In one optional way, the shard relationship data corresponding to each collection moment can be directly stored, and the shard relationship data is extracted when data is traced back. In another optional way, the shard relationship data can be stored based on the shard relationship rule to obtain the shard relationship data when data is traced back. In still another optional way, the shard relationship data has metadata, that is, the data corresponding to the shard relationship includes the shard relationship data and third metadata, and the management node can store the third metadata to obtain the shard relationship data based on the metadata when data is traced back. The third metadata is used to describe the shard relationship data. When data is traced back, the shard relationship data is obtained based on the third metadata of the shard relationship data. The shard relationship data is obtained based on the third metadata, which can realize fast obtaining of the shard relationship data. In this way, only the third metadata needs to be stored when data is not traced back, and the shard relationship data is restored based on the third metadata when the shard relationship data is needed, which reduces the occupation of the storage space.

[0163] The management node stores the backup data of the database system in the local storage space or the cloud storage space of the database, and the backup data includes the shard relationship data of different periods in the database system. The management node directly extracts the data at the corresponding position in the backup data based on the third metadata to obtain the shard relationship data.

[0164] In one optional way, the management node obtains the metadata of the shard relationship data corresponding to the h third collection moments, which can be performed before S505. h is a positive integer. Alternatively, the management node obtains the metadata of the shard relationship data corresponding to the h third collection moments through h times of metadata collection processing. The metadata of the shard relationship data can be collected periodically or after receiving a data collection instruction. The n first collection moments and the h third collection moments have a one-to-many correspondence or a one-to-one correspondence. The third metadata of the shard relationship data is the metadata corresponding to the target historical moment, which is the metadata corresponding to the target historical moment filtered from the metadata corresponding to the h third collection moments by the management node. The acquisition process of the third metadata of the shard relationship data can refer to the acquisition process of the second metadata, which is not described herein.

[0165] In the two optional manners, the metadata of the sharded relationship data is collected in association with the metadata of the log data and the metadata of the sharded data. In the foregoing S603, the metadata corresponding to each of the k collection time points includes the metadata of the log data, the metadata of the plurality of sharded data, and the metadata of the sharded data. Figure 8 is a timeline diagram of another data rollback method provided by an embodiment of the present application. As shown in Figure 8 , assuming that a snapshot of the metadata of the sharded relationship data obtained by using the snapshot technology is referred to as a third snapshot, the collection time period of the metadata corresponding to a collection time point includes a metadata collection time period L1 of the log data, a metadata collection time period L2 of the sharded data, and a collection time period L3 of the sharded relationship data. The start time of the collection time period is the acquisition time of the second snapshot, and the end time of the collection time period is the acquisition time of the third snapshot.

[0166] In an optional example, the management node updates the plurality of sharded data of the target historical time based on the sharded relationship data and the log data of the target historical time, so that the updated plurality of sharded data is consistent with the log data. The process includes:

[0167] A1, based on the data end position indicated by the first metadata of the log data and the data end position indicated by the second metadata of the plurality of sharded data, determining the incremental log data.

[0168] As shown in Figure 7 , in a collection time period, incremental log data may be generated due to the writing of data. The incremental log data is the data of the log data relative to the plurality of sharded data. The management node can obtain the difference data by subtracting the data end position indicated by the second metadata of the plurality of sharded data from the data end position indicated by the first metadata of the metadata of the log data obtained in the collection time period. The difference data is the incremental log data.

[0169] In an optional implementation manner, the data end position indicated by the first metadata of the log data is represented by a continuous landing LSN, which is the largest LSN among the LSNs corresponding to the log data that has been continuously landed at the corresponding time; and the data end position indicated by the second metadata of the plurality of sharded data is represented by a global sharded landing LSN, which is the largest LSN among the LSNs corresponding to the sharded data that has been globally continuously landed at the corresponding time.

[0170] The continuous landing LSN is the largest LSN among the LSNs corresponding to the log data that has been continuously landed at the corresponding time (for example, the target historical time). The continuously landed means landed and continuously landed. In the embodiment of the present application, the landed data of the log data can be obtained first, and the continuously landed data in the obtained landed data is taken as the continuously landed log data.

[0171] wherein one log data has been persisted refers to each copy of the log data has been stored. For the same log data of the database, due to the difference of the storage medium of the writing, the writing speed is also different, and thus the continuous persistence LSN is constantly changing.

[0172] Figure 9 An exemplary storage state diagram of the log data of the target historical moment is provided in the embodiments of the present application. Figure 9 wherein the copies 1-3 of the log data need to be respectively stored in the data storage nodes A-C, and the LSN of the transaction record in the log data is LSN700-LSN1000. It is assumed that the copy 1 of the log data of the target historical moment has been written into the data of the data storage node A with the LSN of LSN700-LSN750, the copy 2 of the log data has been written into the data of the data storage node B with the LSN of LSN700-LSN720, and the copy 3 of the log data has been written into the data of the data storage node C with the LSN of LSN700-LSN800. Then, the LSN corresponding to the continuously persisted log data in the data storage nodes A-C is LSN700-LSN720. Among them, the maximum LSN is LSN720. Thus, the continuous persistence LSN of the log data is LSN720.

[0173] Since the sharded log data is obtained based on the log data, the data structure thereof is the same as that of the log data, and carries the LSN; and the sharded page log data is obtained by converting the corresponding sharded log data, and also carries the LSN of the corresponding sharded log data. Therefore, the LSN is carried in the log data and the sharded data of the transaction log. The global sharded persistence LSN is determined based on the LSN in the sharded data.

[0174] wherein the global sharded persistence LSN is the maximum LSN among the LSNs corresponding to the sharded data that has been globally continuously persisted at the corresponding moment (for example, the target historical moment). Among them, having been globally continuously persisted refers to having been persisted and globally continuously persisted. In the embodiments of the present application, the data that has been persisted of each sharded data can be obtained first, and then the continuously persisted data in the obtained data that has been persisted is obtained as the sharded data that has been globally continuously persisted. It is worth noting that the sharded page data is obtained by converting the sharded log data after the sharded log data is persisted, that is, the carried LSN is the same as the LSN carried by the corresponding sharded log data. Therefore, the aforementioned global sharded persistence LSN can be the maximum LSN among the LSNs corresponding to the sharded log data that has been globally continuously persisted at the corresponding moment, without considering the LSN in the sharded page data.

[0175] One (or one piece) of the slice data has been written to disk means that each copy of the slice data has been stored. For the same slice data of the database, due to the difference of the storage medium, the writing speed is also different, so the data written to disk will change, and the corresponding LSN will also change.

[0176] For example, Figure 10 For example, Figure 10 The embodiment of the present application provides a schematic storage state diagram of the slice data of the target historical moment. It is assumed that the copies 1 to 3 of the slice log data 1 are stored in the data storage nodes D, E and G respectively. After data rollback, the LSN of the data written to the data storage node D by the copy 1 of the slice log data 1 is LSN100 to LSN300, the LSN of the data written to the data storage node E by the copy 2 of the slice log data 1 is LSN100 to LSN140 and LSN160 to LSN170, and the LSN of the data written to the data storage node E by the copy 3 of the slice log data 1 is LSN100 to LSN140 and LSN171 to LSN180. Then the LSN of the transaction record of the slice log data 1 written to disk is LSN100 to LSN140. Similarly, Figure 10 the LSN of the transaction record of the slice log data 2 written to disk is LSN301 to LSN350, Figure 10 and the LSN of the transaction record of the slice log data 3 written to disk is LSN401 to LSN460.

[0177] Continuous writing to disk means that the data is written to disk in ascending order of LSN. Once the LSN is interrupted in the middle, the following data is not continuous data. Global continuous writing to disk means that all slice data is continuously written to disk. That is, all slice data is written to disk in ascending order of LSN.

[0178] Please continue to refer to Figure 10 , the LSN of the transaction record of the slice log data 1 to 3 written to disk is LSN100 to LSN140, LSN301 to LSN350 and LSN401 to LSN460 respectively. Among them, although for the slice log data 2, the transaction log from LSN301 to LSN320 can be regarded as continuous writing to disk, for all slice log data, that is, the slice log data 1 to 3, there is a break in the LSN between LSN140 and LSN301, so the LSN of the global continuous writing to disk data is LSN100 to LSN140. Among them, the maximum LSN is LSN140, so the global slice writing to disk LSN is LSN140.

[0179] Referring to the foregoing S502, the management node determines a plurality of shard data based on the log data through the shard relationship data. In an optional manner, the shard data (e.g., shard log data) is the same as the LSN of the log data to which the shard data belongs, that is, the LSN carried by the transaction record in the shard data is not changed when the shard data is written into the corresponding shard; in another optional manner, the shard data is different from the LSN of the log data to which the shard data belongs, that is, the LSN carried by the transaction record in the shard data is changed (the LSN carried in the shard data and the LSN carried in the log data to which the shard data belongs have a mapping relationship) when the shard data is written into the corresponding shard. In the two optional manners, the correspondence between the LSN of the shard data and the LSN of the log data to which the shard data belongs can be recorded by establishing a data correspondence relationship, such as being completely the same or having a mapping relationship.

[0180] The management node can determine a target LSN corresponding to the global shard landing LSN in the log data based on the data correspondence relationship, and then determine the incremental log data based on the target LSN and the continuous landing LSN, wherein the LSN corresponding to the incremental log data is between the target LSN and the continuous landing LSN. Please refer to Figure 9 and Figure 10 , assuming that the shard data is the same as the LSN of the log data to which the shard data belongs, the target LSN is equal to the global shard landing LSN, which is LSN 140. The continuous landing LSN is LSN 720. Then the LSN corresponding to the incremental log data is LSN 140 to LSN 720, and the transaction record in which LSN 140 to LSN 720 is located is determined as the incremental log data.

[0181] A2, the management node determines at least one incremental shard data based on the incremental log data and the shard relationship data.

[0182] Optionally, referring to the foregoing S502, assuming that the shard relationship data includes the first correspondence relationship and the second correspondence relationship. The management node can perform shard processing on the incremental log data based on the foregoing shard relationship data. For example, for each sub-data of the obtained incremental log data, the management node can query the first correspondence relationship to obtain the database object corresponding to the sub-data; then, the second correspondence relationship is queried by using the database object corresponding to the sub-data to obtain the corresponding shard, (i.e., determining the shard corresponding to each sub-data, which can be regarded as mapping each sub-data into the corresponding shard). Finally, the management node determines the sub-data corresponding to the same shard as one shard incremental log data. Thus, at least one incremental shard data is obtained.

[0183] A3, the management node updates the plurality of shard data based on the shard relationship data and the at least one incremental shard data, and the updated plurality of shard data includes the at least one incremental shard data.

[0184] For example, at least one incremental shard data is incremental shard data generated in the collection time period corresponding to the target historical moment, and the data amount of the at least one incremental shard data is greater than or equal to the sum of the data amounts of all incremental shard data added after the corresponding metadata is acquired in the collection time period. Correspondingly, the at least one incremental shard data has a possibility of data duplication with the backtracked shard data. Figure 7 Figure 8 For example, at least one incremental shard data is incremental shard data generated in the collection time period corresponding to the target historical moment, and the data amount of the at least one incremental shard data is greater than or equal to the sum of the data amounts of all incremental shard data added after the corresponding metadata is acquired in the collection time period. Correspondingly, the at least one incremental shard data has a possibility of data duplication with the backtracked shard data. Figure 11 is a backtracking principle diagram provided by an embodiment of the present application. As shown in Figure 11 , it is assumed that the start time of the collection time period is the first time, and the end time is the second time. Two shard data obtained by backtracking are shard data 1 and shard data 2. The shard data 1 is obtained by backtracking the second metadata acquired at the third time, and the shard data 2 is obtained by backtracking the second metadata acquired at the fourth time. The incremental shard data corresponding to the shard data 1 (the shard data 1 and the incremental shard data corresponding thereto are stored in the same shard) is Figure 11 , the added data 702 from the third time to the second time in the incremental shard data 601. The incremental shard data corresponding to the shard data 2 (the shard data 2 and the incremental shard data corresponding thereto are stored in the same shard) is Figure 11 , the added data 703 from the fourth time to the second time in the incremental shard data 602. At least one incremental shard data obtained based on the incremental log data is Figure 11 , the added data 701 from the first time to the second time in the incremental shard data 601, which includes the data corresponding to the shard data 1 in the collection time period and the data corresponding to the shard data 2 in the collection time period. As can be seen from Figure 11 , the difference data obtained by subtracting the data 702 from the data 701 and then subtracting the data 703 is the data that will cause the problem of repeated writing.

[0185] Optionally, the update process of the incremental shard data in the embodiment of the present application supports the idempotent property. The update process includes:

[0186] Step A31, for each incremental shard data, detecting whether the data stored in the corresponding target shard has the same data as the incremental shard data.

[0187] The target shard is the shard to which the incremental log data needs to be written, which is determined based on the shard relationship data. For example, the management node can compare the LSN of the transaction record in the data stored in the target shard with the LSN of the transaction record in the incremental shard data. If there is a transaction record with the same LSN, the transaction record with the same LSN is determined as the same data; if there is no transaction record with the same LSN, it is determined that the data stored in the target shard does not have the same data as the incremental shard data.

[0188] ​Step A32: When the data stored in the corresponding target shard is different from the incremental shard data, write the incremental shard data into the corresponding target shard.

[0189] When the data stored in the target shard is different from the data in the incremental shard, it means that there is no duplicate data between the data stored in the target shard and the data in the incremental shard, and the incremental shard data can be written into the corresponding target shard.

[0190] Step A33: When the corresponding target shard stores data that is the same as the incremental shard data, writing incremental shard data is prohibited. Alternatively, incremental shard data can be used to overwrite the data in the corresponding target shard that is the same as the incremental shard data. The target shard is the shard corresponding to the incremental shard data determined based on the shard relationship data.

[0191] When the data stored in the target shard is different from the data in the incremental shard, it means that the data stored in the target shard contains duplicate data. By using the above two methods, duplicate data can be avoided in the final stored shard data, reducing the storage space occupied by redundant data and achieving idempotency.

[0192] In this embodiment, based on the log data at the target historical moment, multiple shard data at the target historical moment are updated, so that the updated multiple shard data are consistent with the log data, ensuring the data consistency of the distributed database and realizing effective data backtracking.

[0193] It is worth noting that the consistency processing in this application embodiment can also be implemented in other ways. For example, after S508 or S606, the management node can restart the database process, such as by executing a crash recovery process. Through this crash recovery process, the log data of the target historical moment is used to update multiple shard data of the target historical moment, which can also ensure the stability and data consistency of the database system after restart.

[0194] Figure 12 This is a schematic diagram of the structure of a database system provided in an embodiment of this application. This database system is used to execute the aforementioned data backtracking method. Figure 12 As shown, Figure 12 Taking the acquisition of metadata via snapshots (e.g., the first metadata and the second metadata corresponding to the target historical moment are acquired via snapshots) as an example, this database system includes a management sub-node, a backtracking sub-node, a computation layer, and a storage layer. The functions of the management sub-node and the backtracking sub-node can be referred to the preceding explanation. Figure 3The computing layer includes modules related to the log data, examples of which include a computing layer snapshot flashback module, a data organization management module, a storage module, and a buffer pool. The computing layer snapshot flashback module is configured to flashback the log data under the control of the rollback child node, i.e., perform S507 or S605 described above. The data organization management module is configured to process the data stored in the storage module to obtain high-level semantic data, i.e., data that can be understood by a user. The storage module is configured to store snapshot data, sharding relationship data, and index data (which is used to index table entries in a relational table), etc. The snapshot data can include the first snapshot and / or the third snapshot described above, and the definition of the sharding relationship data is as described in S502 above. The buffer pool is configured to store transaction data.

[0195] The storage layer includes modules related to the sharding log data, examples of which include a storage layer flashback agent module, a plurality of storage layer snapshot flashback modules, a storage layer startup management module, a data organization management module, a storage module, and a buffer pool. The storage layer flashback agent module is configured to manage the plurality of storage layer snapshot flashback modules under the control of the rollback child node, e.g., pass through instructions issued by the rollback child node to the plurality of storage layer snapshot flashback modules. The plurality of storage layer snapshot flashback modules correspond one-to-one to the plurality of shards, and each storage layer snapshot flashback module is configured to flashback the sharding data in the shard, i.e., perform S508 or S606 described above. The storage layer startup management module is configured to control the stopping and starting of the data processing process in the shard, e.g., control the stopping of data writing or the starting of data writing in the shard. In an optional implementation, the storage layer startup management module can control the suspension of data writing in the shard before S508 or S606, and control the restart of data writing in the shard after S508 or S606. The data organization management module is configured to process the data stored in the storage module to obtain high-level semantic data, i.e., data that can be understood by a user. The storage module is configured to store snapshot data, sharding log data, and page data. The snapshot data can include the second snapshot described above, and the definition of the page data is as described in S502 above. The definition of the page data is as described in the foregoing embodiments. The buffer pool is configured to store transaction data.

[0196] The database system architecture provided in the foregoing embodiments separates the computing layer and the storage layer, and the log data and the plurality of sharding data can be rolled back separately, which enables the flashback of the data. After the flashback, consistency processing is performed to ensure the consistency of the computing layer and the storage layer, which can effectively reduce the impact of data rollback on the performance of the database system.

[0197] Figure 13 FIG. 4 is a schematic diagram of a rollback effect according to an embodiment of the present application. Figure 13 For example, the rollback effect of one shard of data is as follows, Figure 13Suppose before the rollback, the sharded data includes sequentially linked files 1 to 5, the current write position of the sharded data is file 5, and the foregoing S508 or S606 is performed on the sharded data to obtain metadata corresponding to one collection time point, for example, the metadata is obtained through a snapshot, then the metadata corresponding to the one collection time point includes metadata of files 1 to 3, Figure 3 Suppose the target historical time point selected by the user is the collection time point, then after the rollback, files 4 and 5 are deleted (indicated by a cross in the figure), files 1 to 3 are retained, and the current write position of the sharded data is updated to file 6 after file 3. The rollback effect of the log data can also be referred to the example of Figure 13 and details are not repeated herein.

[0198] In the embodiments of the present application, the foregoing S503, S504 or S603, the first collection time point, the second collection time point or the collection time point can be represented by a timestamp (such as an atomic clock timestamp) or a globally unique identifier. As long as it can uniquely represent the collected metadata.

[0199] It should be noted that after obtaining the metadata (for example, after obtaining the metadata by using a snapshot, refer to the foregoing S503, S504 or S603), the data corresponding to the metadata (that is, the data from which the metadata is derived) needs to be added with a lock identifier, and the lock identifier is used to indicate that the corresponding data is prohibited from being deleted, that is, to be stored in a fixed manner. In this way, it can be avoided that the data is mistakenly deleted due to some other reasons, resulting in that the effective data rollback cannot be performed. For example, the snapshot is actually a record of the metadata at the snapshot time point, and after the snapshot is generated, the file corresponding to the snapshot will be retained by adding the lock identifier, so as to ensure that the file is not cleared.

[0200] In summary, in the embodiments of the present application, the first metadata corresponding to the target historical time point of the log data is used to rollback the log data, and the second metadata corresponding to the target historical time point of the sharded data is used to rollback the sharded data, without performing the inverse operation of the reading and updating operation of the logical record, the data rollback efficiency is relatively high, and the data rollback delay is effectively reduced.

[0201] For the same database, the rollback delay range of the related technology is usually several minutes to several tens of minutes, and the rollback delay range of the data rollback method provided in the embodiments of the present application is usually several seconds to several tens of seconds. Obviously, the rollback delay of the embodiments of the present application is greatly reduced, and the flashback of the database can be realized.

[0202] In addition, the update operation recorded by using the additional storage space in the related art occupies more storage resources, and generally needs to be stored in the cloud storage space and consumes certain communication resources. However, the embodiment of the present application only needs to record the metadata of the log data and the metadata of the sharded data, the data volume of the metadata is small, the occupied storage resources are less, and the metadata can be stored in the local storage space of the database, thereby reducing the consumption of communication resources. In addition, the data rollback method provided by the embodiment of the present application occupies less storage resources, and has less impact on the performance of the data nodes (especially the data storage nodes) of the database.

[0203] In the related art, the database supporting the data rollback function needs to record the update operation after the initialization of the database, and the limitation on the database is more. The data rollback method provided by the embodiment of the present application can trigger the data rollback function through the user trigger or the pre-set manner, and the setting of the data rollback function is more flexible.

[0204] It should be noted that the order of the steps of the data rollback method provided by the embodiment of the present application can be adjusted appropriately, for example, S503 and S504 can be executed synchronously, and the steps can be increased or decreased according to the situation. Any person skilled in the art can easily think of the changed method within the technical range disclosed by the present application, which should be covered in the protection range of the present application, and therefore will not be described again.

[0205] Figure 14 FIG. 8 is a structural schematic diagram of a data rollback device 80 provided by the embodiment of the present application, as shown in the figure, the device 80 includes: Figure 14

[0206] The first obtaining module 801 is configured to obtain first metadata corresponding to a target historical moment in the metadata of the log data; the second obtaining module 802 is configured to obtain second metadata corresponding to the target historical moment in the metadata of each sharded data in a plurality of sharded data, the plurality of sharded data being data obtained by sharding the log data; the first processing module 803 is configured to process the log data according to the first metadata to obtain the log data at the target historical moment; and the second processing module 804 is configured to process the plurality of sharded data according to the second metadata to obtain the plurality of sharded data at the target historical moment.

[0207] In the embodiment of the present application, the first processing module restores the log data based on the first metadata corresponding to the target historical moment of the log data, and the second processing module restores the sharded data based on the second metadata corresponding to the target historical moment of the sharded data, without performing the inverse operation of the logical recording and the update operation, so that the data rollback efficiency is high and the data rollback delay is effectively reduced.

[0208] ​In an optional mode, the first metadata corresponding to the target historical moment and the second metadata corresponding to the target historical moment are obtained through snapshot.

[0209] In an optional mode, the first processing module 803 is configured to: according to the data position indicated by the first metadata, roll back the log data to obtain log data of the target historical moment.

[0210] In an optional mode, the second processing module 804 is configured to: according to the data position indicated by the second metadata, roll back the plurality of shard data to obtain a plurality of shard data of the target historical moment.

[0211] Figure 15 FIG. 8 is a structural schematic diagram of a data rollback apparatus 80 provided by an embodiment of the present application, as shown in the figure, the apparatus further includes an updating module 805 configured to update the plurality of shard data of the target historical moment based on the log data of the target historical moment. Figure 15

[0212] In an optional mode, the updating module 805 is configured to: update the plurality of shard data of the target historical moment based on shard relationship data and the log data of the target historical moment, the shard relationship data being used to record a corresponding relationship between sub-data in the log data and a shard.

[0213] In an optional mode, the shard relationship data is obtained by restoring third metadata of the shard relationship data.

[0214] In an optional mode, the updating module 805 is configured to: determine incremental log data based on a data end position indicated by the first metadata of the log data and a data end position indicated by the second metadata of the plurality of shard data; determine at least one incremental shard data based on the incremental log data and the shard relationship data; and update the plurality of shard data based on the shard relationship data and the at least one incremental shard data, the updated plurality of shard data including the at least one incremental shard data.

[0215] In an optional mode, the metadata of any data in the log data and the shard data includes: data writing information of each file in at least one file, the at least one file being used to store the any data; and data writing information of any file, used to record a position of the any file in the at least one file and an end position of writing data in the any file.

[0216] Optionally, Figure 16 A possible basic hardware architecture of the computer device provided by an embodiment of the present application is schematically provided. Referring to FIG. 9, Figure 16 the computer device 900 includes a processor 901, a memory 902, a communication interface 903 and a bus 904.​

[0217] The number of processors 901 in the computer device 900 can be one or more, Figure 16 Only one processor 901 is shown. Alternatively, the processor 901 can be a central processing unit (CPU). If the computer device 900 has multiple processors 901, the types of the multiple processors 901 can be different or can be the same. Alternatively, the multiple processors 901 of the computer device 900 can also be integrated into a multi-core processor.

[0218] The memory 902 stores computer instructions and data; the memory 902 can store computer instructions and data required to implement the data rollback method provided in the present application, for example, the memory 902 stores instructions for implementing the steps of the data rollback method. The memory 902 can be any one or any combination of the following storage media: non-volatile memory (such as read-only memory (ROM), solid state disk (SSD), hard disk (HDD), optical disk), volatile memory.

[0219] The communication interface 903 can be any one or any combination of the following devices: network interface (such as Ethernet interface), wireless network card, etc. devices with network access function.

[0220] The communication interface 903 is used for data communication between the computer device 900 and other computer devices or terminals.

[0221] The bus 904 can connect the processor 901 with the memory 902 and the communication interface 903. In this way, the processor 901 can access the memory 902 and also use the communication interface 903 to interact with other computer devices or terminals for data exchange.

[0222] In the present application, the computer device 900 executes computer instructions in the memory 902, so that the computer device 900 implements the data rollback method provided in the present application, or so that the computer device 900 deploys a database system.

[0223] In an exemplary embodiment, a non-transitory computer readable storage medium including instructions, such as a memory including instructions, is also provided, and the above instructions can be executed by a processor of a server to complete the data rollback method shown in various embodiments of the present application. For example, the non-transitory computer readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0224] In the above embodiments, all or part of the data backtracking device can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the data backtracking device can be implemented in the form of a computer program product including one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer can be a general-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer readable storage medium of a computer, or transferred from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions can be transferred from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line) or wireless (such as infrared, wireless, microwave, etc.). The computer readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. integrated with one or more available media. The available media can be magnetic media (such as floppy disk, hard disk, magnetic tape), optical media, or semiconductor media (such as solid state disk) and the like.

[0225] In this application, the terms "first", "second" and "third" are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance. The term "at least one" means one or more, and the term "multiple" means two or more, unless otherwise explicitly limited. A refers to B, which means that A is the same as B or A is a simple transformation of B.

[0226] It should be noted that: the data backtracking device provided by the above embodiments only uses the division of the above functional modules as an example when executing the data backtracking method. In actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the data backtracking device and the data backtracking method provided by the above embodiments belong to the same concept, and the specific implementation process is described in the method embodiment, which will not be repeated here.

[0227] Those skilled in the art can understand that all or part of the steps of the above embodiments can be completed by hardware, or by program to instruct related hardware to complete. The program can be stored in a computer readable storage medium. The storage medium mentioned above can be read only memory, disk or optical disk, etc.

[0228] The above is only an optional embodiment of the present application, and does not limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A data rollback method, characterized by, The method comprises: obtaining first metadata corresponding to a target historical time in metadata of log data; obtaining second metadata corresponding to the target historical time in metadata of each piece of shard data in a plurality of piece of shard data, the plurality of piece of shard data being data obtained by sharding the log data; processing the log data according to a data position indicated by the first metadata to obtain log data of the target historical time; processing the plurality of piece of shard data according to a data position indicated by the second metadata to obtain a plurality of piece of shard data of the target historical time; wherein the first metadata and the second metadata each comprise data writing information of each file in at least one file, the at least one file being used to store any data in the log data and the shard data; the data writing information of any file being used to record a position of the any file in the at least one file and an end position of writing data in the any file.

2. The method of claim 1, wherein, The first metadata corresponding to the target historical time and the second metadata corresponding to the target historical time are obtained by snapshot.

3. The method according to claim 1 or 2, characterized in that, The processing of the log data according to the data position indicated by the first metadata to obtain log data of the target historical time comprises: rewinding the log data according to the data position indicated by the first metadata to obtain the log data of the target historical time.

4. The method according to claim 1 or 2, characterized in that, The processing of the plurality of piece of shard data according to the data position indicated by the second metadata to obtain a plurality of piece of shard data of the target historical time comprises: rewinding the plurality of piece of shard data according to the data position indicated by the second metadata to obtain the plurality of piece of shard data of the target historical time.

5. The method according to claim 1 or 2, characterized in that, The method further comprises: updating the plurality of piece of shard data of the target historical time based on the log data of the target historical time.

6. The method of claim 5, wherein, The updating of the plurality of piece of shard data of the target historical time based on the log data of the target historical time comprises: updating the plurality of piece of shard data of the target historical time based on shard relationship data and the log data of the target historical time, the shard relationship data being used to record a corresponding relationship between sub-data in the log data and a shard.

7. The method of claim 6, wherein, The shard relationship data is obtained by restoring third metadata of the shard relationship data.

8. The method according to claim 6 or 7, characterized in that, The updating of the plurality of piece of shard data of the target historical time based on the shard relationship data and the log data of the target historical time comprises: determining incremental log data based on an end position of data indicated by the first metadata of the log data and an end position of data indicated by the second metadata of the plurality of piece of shard data; determining at least one incremental piece of shard data based on the incremental log data and the shard relationship data; updating the plurality of piece of shard data based on the shard relationship data and the at least one incremental piece of shard data, the updated plurality of piece of shard data comprising the at least one incremental piece of shard data.

9. A data rollback apparatus, characterized by comprising: The device comprises: a first obtaining module configured to obtain first metadata corresponding to a target historical time in metadata of log data; The second obtaining module is configured to obtain second metadata corresponding to the target historical moment in metadata of each piece of the plurality of piece data, the plurality of piece data being data obtained by fragmenting the log data; The first processing module is configured to process the log data according to a data position indicated by the first metadata to obtain log data of the target historical moment; The second processing module is configured to process the plurality of piece data according to a data position indicated by the second metadata to obtain a plurality of piece data of the target historical moment; The first metadata and the second metadata each include data writing information of each file in at least one file, and the at least one file is used to store any data in the log data and the piece data; The data writing information of any file is used to record a position of the any file in the at least one file and an end position of writing data in the any file.

10. The apparatus of claim 9, wherein, The first metadata corresponding to the target historical moment and the second metadata corresponding to the target historical moment are obtained by snapshot.

11. The apparatus of claim 9 or 10, wherein, The first processing module is configured to: back up the log data according to the data position indicated by the first metadata to obtain the log data of the target historical moment.

12. The apparatus of claim 9 or 10, wherein, The second processing module is configured to: back up the plurality of piece data according to the data position indicated by the second metadata to obtain the plurality of piece data of the target historical moment.

13. The apparatus of claim 9 or 10, wherein, The apparatus further includes an updating module configured to update the plurality of piece data of the target historical moment based on the log data of the target historical moment.

14. The apparatus of claim 13, wherein, The updating module is configured to: update the plurality of piece data of the target historical moment based on piece relationship data and the log data of the target historical moment, the piece relationship data being used to record a corresponding relationship between sub data in the log data and a piece.

15. The apparatus of claim 14, wherein, The piece relationship data is obtained by restoring third metadata of the piece relationship data.

16. The apparatus of claim 14 or 15, wherein, The updating module is configured to: determine incremental log data based on an end position of data indicated by the first metadata of the log data and an end position of data indicated by the second metadata of the plurality of piece data; determine at least one incremental piece data based on the incremental log data and the piece relationship data; update the plurality of piece data based on the piece relationship data and the at least one incremental piece data, the updated plurality of piece data including the at least one incremental piece data.

17. A computer device, comprising: include: a processor and a memory; the memory is configured to store computer instructions; the processor is configured to execute the computer instructions stored in the memory, so that the computer device executes the data rollback method in any one of claims 1 to 8.

18. A computer-readable storage medium, characterized in that, The computer readable storage medium includes computer instructions, and the computer instructions instruct the computer device to execute the data rollback method in any one of claims 1 to 8.

Images