A database flashback implementation method based on a value log system

Abstract

The application discloses a database flashback implementation method based on a value log system, and comprises the following steps: a working thread / process receives a user library flashback request; the working thread / process executes a suspended database or data processing system; the suspended working thread stops providing services externally, and the current working thread / process is excepted; an instance is set as a suspended mode; the working thread / process determines a flashback point of the database flashback; the working thread / process truncates the value log based on the determined flashback point, and sets the tail of the value log block to zero, so that the new log reuses the value log storage space; the working thread / process cleans up invalid records in the index based on the determined flashback point, that is, the index records after the flashback point; the working thread / process brings other working threads / processes online, reopens the instance as a normal mode, and returns a user end flashback success. The application supports the flashback to a specified flashback point, does not need additional flashback logs and REDO logs, and the value log supports the library-level flashback originally.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of database, in particular to a database flashback implementation method based on value log system. BACKGROUND

[0002] The library flashback of traditional database is to restore the entire database to a specified time point, which is actually an incomplete recovery of the database. It is realized based on flashback log and REDO log (such as Oracle), and the flashback log records the old version of data page in units of data page. The flashback log has certain limitations, not all old values are recorded, a set of operations is packaged together, similar to a reverse incremental snapshot (a set of data pages), so the library-level flashback needs to repair to the target flashback point with the help of REDO log.

[0003] The traditional database needs to configure the flashback log to support database flashback, needs additional storage space, and only supports database flashback within a limited time period, which is related to the flashback storage space and the archive log. The library-level flashback calculates an earlier time T2 according to the time T1 given by the user and the flashback log, and then fills the flashback log back to the data file in reverse. Finally, through the archive REDO log, the library flashback is completed at the T1 time point. Here, if the data file is expanded, it will not shrink back, and the user needs to shrink the data file back with the help of the recovery tool before executing the library flashback. The flashback support of the traditional data depends on the flashback log, REDO log and real data file, and essentially supports the library flashback through data redundancy, which causes a large waste of storage space. SUMMARY

[0004] The technical problem to be solved by the present application is to provide a database flashback implementation method based on value log system, which supports flashback to a specified flashback point, including any valid cutoff offset address or cutoff LSN or cutoff time point since the initialization library, does not need additional flashback log and REDO log, and supports library-level flashback natively by value log.

[0005] To solve the above technical problems, the present application provides a database flashback implementation method based on value log system, comprising the following steps:

[0006] Step 1: The working thread / process in the database / KV data processing system based on value log technology receives a user library flashback request;

[0007] Step 2: The working thread / process executes the suspended database or data processing system, suspends the working thread to stop providing services to the outside, except the current working thread / process, and sets the instance to the suspended mode;

[0008] Step 3, the working thread / process determines the flashback point of the database flashback, the flashback point including the cutoff offset address, the cutoff LSN and the cutoff time;

[0009] Step 4, the working thread / process truncates the value log based on the flashback point determined in step 3, and zeroes the tail of the value log block, so that the new log reuse value log storage space;

[0010] Step 5, the working thread / process cleans up the invalid records in the index based on the flashback point determined in step 3, that is, the index records after the flashback point;

[0011] Step 6, the working thread / process brings up other working threads / processes, reopens the instance to the normal mode, and returns the user end flashback success.

[0012] Preferably, in step 3, the determination of the flashback point of the database flashback includes the following cases or their combinations:

[0013] (a) database flashback based on the cutoff offset address;

[0014] (b) database flashback based on the cutoff LSN;

[0015] (c) database flashback based on the cutoff time;

[0016] (d) database flashback based on the combination of the cutoff time, the cutoff LSN or the cutoff offset address.

[0017] Preferably, in step (a), the database flashback based on the cutoff offset address is specifically: in the value log technology-based data processing system, data is continuously generated, and the size of the offset directly reflects the new and old degree of data.

[0018] Preferably, in step (b), the database flashback based on the cutoff LSN is specifically: various data processing systems including database / KV systems have a log sequence number LSN flag for each log record, the LSN is monotonically increasing, and continuously increases with the increase of operations on the database; in the value log system, data is a log, so it saves LSN information; after the user specifies the cutoff LSN, the corresponding physical block of the specified cutoff LSN is quickly determined by using binary method according to the LSN information recorded in the block, and further scanning the records in the block in sequence to obtain the offset address corresponding to the physical record corresponding to the specified LSN, which is also the cutoff offset address corresponding to the cutoff LSN.

[0019] Preferably, in step (c), the database flashback based on the cutoff time is specifically as follows: the user specifies the cutoff time in the database flashback request, and when the cutoff time is specified, the database is flashed back to the specified cutoff time, and all records are retained; when the specified cutoff time is to be supported for flashback, the system has multiple optional ways to implement. For example: if the time accuracy requirement is not high, consider writing the system time value in the block header / tail of the value log file, and then use binary search on the data block to determine the physical block corresponding to the specified cutoff time, so as to obtain the corresponding cutoff offset address; insert a time record into a specific time table at a certain time, and according to the cutoff time, the offset address of the physical record corresponding to the cutoff time can be found; if the time accuracy requirement is high, consider retaining the system time information when the data is inserted in the physical record, and then use binary search at the data block level according to the cutoff time (for example, a total of 100 blocks, then take the first physical record of the 50th block, if the time is greater than the cutoff time, then access the 25th block next time, continue to take the first physical record of the block inserting_time for comparison, and so on) to find the data block corresponding to the cutoff time, and then sequentially scan the data block to find the physical record closest to and greater than or equal to the cutoff time to obtain the offset address of the physical record corresponding to the cutoff time. Once the physical record closest to and greater than or equal to the cutoff time is found, the offset address of the physical record is known, that is, the cutoff offset address.

[0020] Preferably, in step (d), the combination based on the cutoff time, the cutoff LSN, or the cutoff offset address is specifically as follows: if there are multiple dimensions, then they are calculated respectively, and then the earliest cutoff offset address is selected as the flashback point; if multiple dimensions are specified, then the cutoff offset address C and the cutoff offset address B are calculated according to the methods of steps (c) and (b) respectively, and the earlier one is selected as the cutoff offset address, that is, the flashback point; if the user specifies a fuzzy cutoff time (such as only to the day, lacking the time within the day), and the user also specifies the cutoff LSN, then the offset address corresponding to the cutoff LSN within the day is selected as the flashback point, and a more accurate strategy is used.

[0021] Preferably, in step 4, according to the cutoff offset address of the flashback point, the cutoff offset address is "virtualized" to the tail, when the value log record is obtained, the "virtualized" address space is hit, and then an empty record is returned, and when the value log space is recycled, the "virtualized" address space is recycled; the target address space is defined as an invalid address, which is called virtualization.

[0022] Preferably, in step 5, based on the flashback point (which can be the cutoff offset, the cutoff LSN, or the cutoff time), the index tree merging operation is actively initiated. During the merging process, if an index is found to point to a data record after the flashback point, the index entry is discarded directly, and the generated new index tree will not contain index records after the flashback point.

[0023] Preferably, in step 5, a virtual address is set, and index entries in the "virtual" address space are filtered out when using the index for retrieval, and the "virtual" index entries are reclaimed when the index is merged.

[0024] The beneficial effects of this invention are as follows: This invention supports flashback to specified flashback points, which include any valid cutoff offset address, cutoff LSN, or cutoff time point since the initialization of the database. Compared with the database-level flashback of traditional databases, the database flashback based on the value log system does not consume flashback log storage space or REDO archive log storage space. Attached Figure Description

[0025] Figure 1 This is a schematic diagram of the method flow of the present invention. Detailed Implementation

[0026] like Figure 1 As shown, a database flashback implementation method based on a value log system includes the following steps:

[0027] Step 1: In a database / KV data processing system based on value logging technology, worker threads / processes receive user database flashback requests;

[0028] Step 2: The worker thread / process suspends the database or data processing system, and the suspended worker thread no longer provides services to the outside world (except for the current worker thread / process). Set the instance to suspension mode.

[0029] Step 3: The worker thread / process determines the flashback point of the database flashback (including the cutoff offset address / cutoff LSN / cutoff time);

[0030] Step 4: Based on the flashback point determined in Step 3, the worker thread / process truncates the value log and places it at the end of the zero value log block so that new logs can reuse the value log storage space.

[0031] Step 5: The worker thread / process cleans up invalid records in the index (i.e., indexes that are recorded after the flashback point) based on the flashback point determined in Step 3;

[0032] Step 6: Launch other worker threads / processes, reopen the instance in normal mode, and return a success flashback to the user.

[0033] Example 1:

[0034] The following example assumes that the keyword of the library flashback is flashback (indicating that it contains flashback), followed by offset (indicating the flashback cutoff offset address) / lsn (indicating the flashback cutoff LSN) / timestamp (indicating the flashback time point).

[0035] The column information of a table account in a value log system is shown in Table 1, the page size of the system is assumed to be 10000 bytes, and the current size of the value log is assumed to be 1000000:

[0036] Table 1 Column information of the account table

[0037] Column name Column type Column meaning id varchar(20) Account id, primary key, contains system time information when record is inserted in index name varchar(20) Account name balance int Account balance

[0038] Inserting 3 records in the table account is shown in Table 2:

[0039] Table 2 Data operation history of the account table

[0040] Operation time Operation type Corresponding SQL 2021-07-01 13:00:00 Insert insert into account values('A001','Zhang San',110); 2021-07-15 14:00:00 Insert insert into account values('A002','Li Si',120); 2021-07-16 17:30:00 Insert insert into account values('A003','Wang Wu',130);

[0041] After the insertion, the user queries all the data information, data offset address, LSN information and insertion time stamp (the physical record contains the insertion time) in the table account through the following statement:

[0042] Select offset,lsn,inserting_time,id,name,balance from account;

[0043] The query result is shown in Table 3:

[0044] Table 3 Data information of the account table

[0045] offset lsn Inserting_time id name balance 640100 101 2021-07-01 13:00:00 A001 Zhang San 110 640200 102 2021-07-15 14:00:00 A002 Li Si 120 640300 103 2021-07-16 17:30:00 A003 Wang Wu 130

[0046] Under the above assumptions, it can be known that the data of the account table is on the 65th page, and the specific steps of the method for performing database flashback are as follows:

[0047] S101: The user executes the following database flashback statement on the client:

[0048] Flashback database to timesamp'2021-07-01 10:00:00.000';

[0049] The database / KV data processing system receives the flashback request.

[0050] S102: The working thread / process executes the suspend instance operation, i.e. suspends other working threads.

[0051] S103: The working thread / process determines the value log offset address 640200 corresponding to the flashback point according to the flashback point (including the cutoff time point).

[0052] S104: The working thread / process truncates the value log file according to the value log offset address 640200 located by the flashback point in S103, and obviously the corresponding A002 second record and A003 third record in the table are deleted.

[0053] Preferably, the value log file is not truncated, and the "virtualized" address space (640200, 1000000) is recorded to the value log description file. When the data record is acquired, if the "virtualized" address space is hit, an empty record Nil is returned.

[0054] S105: The working thread / process executes the index item removal operation for the index tree, which removes the data record with the address located at (640200, 1000000).

[0055] Preferably, the index item removal operation for the record with the address located at (640200, 1000000) is delayed, and is removed when the index is naturally merged.

[0056] S106: The working thread / process resets other working threads, sets the system to the normal mode, provides services to the outside, and then returns to the user.

Claims

1. A database flashback implementation method based on a value log system, characterized in that, Includes the following steps: Step 1: The worker thread or process in the database or KV data processing system based on value logging technology receives the user database flashback request; Step 2: The worker thread or process suspends the database or data processing system, stops providing services to the outside world, except for the current worker thread or process, and sets the instance to suspension mode; Step 3: The worker thread or process determines the flashback point of the database flashback. The flashback point includes the cutoff offset address, the cutoff LSN, and the cutoff time. Step 4: Based on the flashback point determined in Step 3, the worker thread or process truncates the value log and places it at the end of the zero-value log block so that the new log can reuse the value log storage space; according to the cutoff offset address of the flashback point, the cutoff offset address is virtualized to the end. When a record of the value log is retrieved, if the virtualized address space is hit, an empty record is returned, and the virtualized address space is reclaimed when the value log space is reclaimed; defining the target address space as an invalid address is called virtualization; Step 5: The worker thread or process cleans up invalid records in the index based on the flashback point determined in Step 3, that is, the index of records located after the flashback point; Step 6: Launch other worker threads or processes, reopen the instance in normal mode, and return a successful flashback to the user.

2. The database flashback implementation method based on a value log system as described in claim 1, characterized in that, In step 3, determining the flashback point of the database flashback includes the following situations or combinations thereof: (a) Database flashback based on cutoff offset address; (b) Database flashback based on cutoff LSN; (c) Perform database flashback based on deadline; (d) Based on a combination of cutoff time, cutoff LSN, or cutoff offset address.

3. The database flashback implementation method based on a value log system as described in claim 2, characterized in that, In step (a), database flashback based on the cutoff offset address is specifically as follows: In a data processing system based on value log technology, data is continuously appended and generated, and the size of the offset directly reflects the newness or oldness of the data.

4. The database flashback implementation method based on a value log system as described in claim 2, characterized in that, In step (b), the database flashback based on the cutoff LSN is specifically as follows: various data processing systems, including databases or KV systems, have a log sequence number (LSN) for each log record. This LSN is monotonically increasing and continuously increases as the number of operations on the database increases. In a value log system, data is logs, so LSN information is stored. After the user specifies the cutoff LSN, the physical block corresponding to the specified cutoff LSN is quickly determined by binary search based on the LSN information recorded in the data block. Then, the records within the block are sequentially scanned to obtain the offset address corresponding to the physical record of the specified LSN. This address is the cutoff offset address corresponding to the cutoff LSN.

5. The database flashback implementation method based on a value log system as described in claim 2, characterized in that, In step (c), database flashback based on the deadline is specifically as follows: the user specifies the deadline in the database flashback request. Specifying the deadline indicates that the database flashback will return to the specified deadline, and all records will be retained. When supporting flashback based on a specified deadline, the system has several optional methods to implement this. If the time precision requirement is not high, consider writing the system time value only at the beginning or end of the value log file, and then using a binary search method to determine the physical block corresponding to the specified deadline, thereby obtaining the corresponding deadline offset address. Insert a time record into a specific time table periodically, and the offset address of the physical record corresponding to the deadline can be found based on the deadline. If the time precision requirement is high, consider retaining the system time information when the data was inserted in the physical record, and then using a binary search method at the data block level to find the data block corresponding to the deadline based on the deadline. Then, sequentially scan the data block to find the physical record that is closest to and greater than or equal to the deadline, which can also obtain the offset address of the physical record corresponding to the deadline. Once the physical record that is closest to and greater than or equal to the deadline is found, the offset address of the physical record is known, which is the deadline offset address.

6. The database flashback implementation method based on a value log system as described in claim 2, characterized in that, In step (d), the combination of cutoff time, cutoff LSN, or cutoff offset address is specifically as follows: if there are multiple dimensions, they are calculated separately, and then the earliest cutoff offset address is selected as the flashback point; if the user-specified cutoff time is ambiguous, and a cutoff LSN is specified at the same time, the offset address corresponding to the cutoff LSN within the day will be selected as the flashback point.

7. The database flashback implementation method based on a value log system as described in claim 1, characterized in that, In step 5, based on the flashback point, the index tree merging operation is initiated. During the merging process, if an index is found to point to a data record after the flashback point, the index entry is discarded directly. The resulting new index tree will not contain any index records after the flashback point.

8. The database flashback implementation method based on a value log system as described in claim 1, characterized in that, In step 5, a virtual address is set, and index entries in the virtual address space are filtered out when using the index for retrieval, and the virtual index entries are reclaimed when the index is merged.

Images