A database transaction management method and device, and a recovery method and device

Abstract

This paper provides a database transaction management method and device, and a recovery method and device, which relate to the field of automatic programming. The database transaction management method includes: when processing the current read transaction, according to the most recent transaction identifier of the current read transaction and the first preset identifier generation rule, determine the virtual identifier of the current read transaction, wherein the virtual identifier of the current read transaction includes the most recent write transaction identifier and sub-identifier of the current read transaction; scan each write transaction record in the transaction log, and determine whether the write transaction record is visible to the current read transaction according to the write transaction identifier in the write transaction record, the virtual identifier of the current read transaction, and the isolation level of the current read transaction. If the write transaction record is visible to the current read transaction, read the data block corresponding to the write transaction record from the database. This paper reduces the amount of data in the transaction log to a great extent, and can quickly recover the transaction by scanning the transaction log when a database failure occurs.

Description

Technical Field

[0001] The present invention relates to the field of automatic programming, and in particular to a database transaction management method and device, and a recovery method and device. Background Art

[0002] Transaction is an important concept in database system. Transaction in database system is a set of operations that constitute a single logical unit of work. It can contain one or more operations, and these operations are logically a whole. These database operations are either all executed successfully or not executed at all. Database transaction is a sequence of one or more operations distributed on corresponding nodes in the network.

[0003] In actual use, distributed database systems may encounter a series of unexpected failures that may prevent transactions from being committed or rolled back normally. These failures usually include: 1) system failures such as power outages, hardware failures such as CPU, operating system failures, abnormal process exits, etc.; 2) media failures such as external memory; 3) computer viruses, etc.

[0004] When the distributed database in the prior art registers logs, transactions are not distinguished. Both write transactions and read transactions need to establish transaction logs, which will result in a large amount of transaction log data. When the database fails and data is restored based on the transaction log, there is a problem that transaction recovery takes a long time. Summary of the invention

[0005] This article aims to solve the problem in the prior art that transaction logs have a large amount of data and that transaction recovery takes a long time when data is restored based on transaction logs in the event of a database failure.

[0006] In order to solve the above technical problems, the first aspect of this article is a database transaction management method, including:

[0007] When processing the current read transaction, determining the virtual identifier of the current read transaction according to the most recent transaction identifier of the current read transaction and the first preset identifier generation rule, wherein the virtual identifier of the current read transaction includes the most recent write transaction identifier and sub-identifier of the current read transaction;

[0008] Scan each write transaction record in the transaction log, and determine whether the write transaction record is visible to the current read transaction according to the write transaction identifier in the write transaction record, the virtual identifier of the current read transaction, and the isolation level of the current read transaction; if the write transaction record is visible to the current read transaction, read the data block corresponding to the write transaction record from the database.

[0009] As a further embodiment of this invention, the database transaction management method further includes:

[0010] When processing the current write transaction, determining the current write transaction identifier according to the most recent historical write transaction identifier of the current write transaction and a second preset identifier generation rule;

[0011] Writing the data page related to the current write transaction into the data buffer of the memory;

[0012] According to the current write transaction, the data page in the data buffer is modified, and the first write data service refreshes the modified data page to the data area of ​​the database;

[0013] According to the current write transaction identifier and the modified data page, a write transaction record is constructed in the transaction log buffer of the memory, and the second write data service refreshes the write transaction record to the transaction log of the database.

[0014] As a further embodiment of the present invention, after the database transaction determines the current write transaction identifier, a start transaction record is generated in a transaction log buffer of the memory, and the third write data service refreshes the start transaction record to the transaction log of the database;

[0015] After the write transaction record is refreshed into the transaction log of the database, a commit transaction record is generated in the transaction log buffer of the memory, and the fourth write data service refreshes the commit transaction record into the transaction log of the database.

[0016] As a further embodiment of the present invention, the write transaction record of the database transaction includes at least one tuple record, and each tuple record includes at least a write transaction identifier, a row identifier, all column data and a mark bit;

[0017] The flag bit is used to identify the write operation type. When the flag bit indicates valid, the corresponding operation type is an update operation or a new operation; when the flag bit indicates invalid, the corresponding operation type is a delete transaction.

[0018] As a further embodiment of the present invention, the write transaction record of the database transaction includes at least one tuple record, each tuple record includes: a write transaction identifier, a row identifier, difference data, and a mark bit, wherein the difference data includes a copy item, a deletion item, and / or a new item;

[0019] The flag bit is used to identify the write operation type. When the flag bit indicates valid, the corresponding operation type is an update operation or a new operation; when the flag bit indicates invalid, the corresponding operation type is a delete transaction.

[0020] The copy item is used to record the bytes occupied by unchanged data, the delete item is used to record the bytes occupied by deleted data, and the new item is used to record modified data;

[0021] The copied items, deleted items and / or newly added items are arranged in the order of all columns corresponding to the row identifiers.

[0022] As a further embodiment of this invention, tuple records of the same row identifier of a database transaction are linked by pointers.

[0023] As a further embodiment of the present invention, after the data page related to the current write transaction is written into the data buffer of the memory, an exclusive lock is added to the data page;

[0024] After constructing the write transaction record, the exclusive lock is released.

[0025] As a further embodiment of the present invention, the first preset identifier generation rule includes a first identifier generation sub-rule and a second identifier generation sub-rule;

[0026] Determining the virtual identifier of the current read transaction according to the most recent historical transaction identifier of the current read transaction and a first preset identifier generation rule includes:

[0027] If the most recent historical transaction of the current read transaction is a write transaction, generating a sub-rule according to the most recent historical write transaction identifier of the current read transaction and the first identifier to determine the virtual identifier of the current read transaction;

[0028] If the most recent historical transaction of the current read transaction is a read transaction, a sub-rule is generated according to the most recent historical read transaction identifier of the current read transaction and a second identifier to determine a virtual identifier of the current write transaction.

[0029] As a further embodiment of the present invention, determining the virtual identifier of the current read transaction according to the identifier of the most recent historical write transaction of the current read transaction and the first identifier generation sub-rule includes:

[0030] Generate a sub-rule using the first identifier, generate a sub-identifier and concatenate the most recent historical write transaction identifier and the sub-identifier of the current read transaction;

[0031] The concatenated identifier is used as the virtual identifier of the current read transaction. As a further embodiment of the present invention, the virtual identifier of the current write transaction is determined according to the historical read transaction identifier most recent to the current read transaction and the second identifier generation sub-rule, including:

[0032] Generate a sub-rule using the second identifier to adjust the sub-identifier in the most recent historical read transaction identifier of the current read transaction;

[0033] The updated identifier is used as the virtual identifier of the current read transaction.

[0034] As a further embodiment of the present invention, determining whether the write transaction record is visible to the current read transaction according to the write transaction identifier in the write transaction record, the virtual identifier of the current read transaction, and the isolation level of the current read transaction includes:

[0035] Determine whether the write transaction identifier in the write transaction record is less than the virtual identifier of the current read transaction;

[0036] Determine whether the tuple record in the write transaction record is the latest record corresponding to the row identifier;

[0037] If the judgment results are all yes, determining whether the tuple record in the write transaction record is visible to the current read transaction according to the isolation level of the current read transaction;

[0038] If the judgment result is no, it is determined that the tuple record in the write transaction record is not visible to the read transaction.

[0039] A second aspect of the present invention provides a database transaction recovery method, wherein the transaction log is recorded using the method described in the above embodiment, and the transaction recovery process when a system failure occurs includes:

[0040] Scan the write transaction records in the transaction log forward and perform redo operations on committed transactions and rolled back transactions.

[0041] As a further embodiment of the present invention, if the write transaction record includes at least: a write transaction identifier, a row identifier, difference data, and a mark bit, wherein the difference data includes a copy item, a deletion item, and / or a new item, performing a redo operation on a committed transaction and a rolled back transaction includes:

[0042] For each committed transaction and each rolled back transaction, find the first tuple record corresponding to each tuple record in the committed transaction and the rolled back transaction;

[0043] Restore the modified data according to each tuple record in the committed transaction and its corresponding first tuple record;

[0044] Restore the modified data according to each tuple record in the rolled back transaction and its corresponding first tuple record;

[0045] Write the modified data into the data area of ​​the database.

[0046] A third aspect of the present invention further provides a database transaction management device, comprising:

[0047] a read transaction processing module, configured to determine, when processing a current read transaction, a virtual identifier of the current read transaction according to a most recent transaction identifier of the current read transaction and a first preset identifier generation rule, wherein the virtual identifier of the current read transaction includes a most recent write transaction identifier and a sub-identifier of the current read transaction;

[0048] A scanning module is used to scan each write transaction record in the transaction log, and determine whether the write transaction record is visible to the current read transaction according to the write transaction identifier in the write transaction record, the virtual identifier of the current read transaction, and the isolation level of the current read transaction; if the write transaction record is visible to the current read transaction, read the data block corresponding to the write transaction record from the database.

[0049] A fourth aspect of the present invention provides a database transaction recovery device, comprising:

[0050] The scanning module is used to forward scan the write transaction records in the transaction log when a system failure occurs, and perform redo operations on submitted transactions and rolled back transactions, wherein the write transaction records are recorded using the database transaction management method described in any of the aforementioned embodiments.

[0051] A fifth aspect of the present invention provides a computer device, comprising a memory, a processor, and a computer program stored in the memory, wherein when the computer program is executed by the processor, the computer program executes instructions of the database transaction management method described in any of the aforementioned embodiments.

[0052] A sixth aspect of the present invention provides a computer storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor of a computer device, the computer program executes instructions of the database transaction management method described in any of the aforementioned embodiments.

[0053] The seventh aspect of the present invention further provides a computer-readable program. When a processor in a computer executes the program, the program executes instructions of the database transaction management method described in any of the aforementioned embodiments.

[0054] The database transaction management method and device, transaction recovery method and device provided in this article, when processing a read transaction, assign a virtual identifier to the read transaction, do not generate a transaction record for the read transaction, generate a write transaction record for the write transaction, and determine whether the write transaction record is visible to the current read transaction based on the write transaction identifier in the write transaction record, the virtual identifier of the current read transaction, and the isolation level of the current read transaction, thereby completing the data reading operation. Because most transactions in actual applications are read transactions, the above technical solution can greatly reduce the amount of data in the transaction log, and when a database failure occurs, the transaction can be quickly recovered by scanning the transaction log.

[0055] In order to make the above and other purposes, features and advantages of this article more obvious and easy to understand, the following specifically cites preferred embodiments and describes them in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the technical solutions in the embodiments of this article or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of this article. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0057] Figure 1 A first flow chart of the database transaction management method according to the embodiment of this invention is shown;

[0058] Figure 2 A first schematic diagram of a tuple record in an embodiment of this invention is shown;

[0059] Figure 3 A schematic diagram showing a transaction record writing method according to an embodiment of the present invention is shown;

[0060] Figure 4 A second schematic diagram of a tuple record according to an embodiment of the present invention is shown;

[0061] Figure 5 A second flow chart of the database transaction management method according to the embodiment of this invention is shown;

[0062] Figure 6 A schematic diagram of a transaction log in an embodiment of this invention is shown;

[0063] Figure 7 A flowchart showing a process of determining whether a write transaction record is visible to a current read transaction according to an embodiment of the present invention;

[0064] Figure 8 A flowchart showing an embodiment of the present invention performing redo operations on a committed transaction and a rolled back transaction;

[0065] Fig. 9 A first structural diagram of a database transaction management device according to an embodiment of the present invention is shown;

[0066] Fig.10 A second structural diagram of the database transaction management device according to the embodiment of this invention is shown;

[0067] Fig.11 A structural diagram of a database transaction recovery device according to an embodiment of this invention is shown;

[0068] Fig.12 A schematic diagram of the Redo log format of an embodiment of this invention is shown;

[0069] Fig.13A structural diagram of a computer device according to an embodiment of the present invention is shown;

[0070] Description of the accompanying symbols:

[0071] 210. Write transaction identifier;

[0072] 220, row identification;

[0073] 230, all column data;

[0074] 240, mark bit;

[0075] 310. Write transaction identifier;

[0076] 320, row identification;

[0077] 330. Difference data;

[0078] 340, mark position;

[0079] 331. Copy item;

[0080] 332. Delete item;

[0081] 333. New items;

[0082] 910, read transaction processing module;

[0083] 920, scanning module;

[0084] 930. Write transaction processing module;

[0085] 1110, scanning module;

[0086] 1120. Rework module;

[0087] 1302. Computer equipment;

[0088] 1304, processor;

[0089] 1306. Memory;

[0090] 1308, driving mechanism;

[0091] 1310, input / output module;

[0092] 1312. Input devices;

[0093] 1314. Output device;

[0094] 1316. Presentation equipment;

[0095] 1318. Graphical user interface;

[0096] 1320, network interface;

[0097] 1322, communication link;

[0098] 1324. Communication bus. DETAILED DESCRIPTION

[0099] The following will be combined with the drawings in the embodiments of this article to clearly and completely describe the technical solutions in the embodiments of this article. Obviously, the described embodiments are only part of the embodiments of this article, not all of the embodiments. Based on the embodiments of this article, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this article.

[0100] It should be noted that the terms "first", "second", etc. in the specification and claims of this article and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of this article described here can be implemented in an order other than those illustrated or described here. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, device, product or equipment that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0101] This specification provides method operation steps as described in the embodiments or flow charts, but more or fewer operation steps may be included based on routine or non-creative work. The order of steps listed in the embodiments is only one way of executing the order of many steps and does not represent the only execution order. When the system or device product is executed in practice, it can be executed in the order of the method shown in the embodiments or the drawings or in parallel.

[0102] It should be noted that the database transaction management method and device, recovery method and device in this article can be used for databases in the financial field, and can also be used for databases in any field other than the financial field. The application field of the database transaction management method and device, recovery method and device in this article is not limited.

[0103] In one embodiment of this invention, Figure 1 As shown, a database transaction management method is provided to solve the problem in the prior art that the transaction log has a large amount of data and when the database fails and the data is restored based on the transaction log, the transaction recovery takes a long time. Specifically, the database transaction management method includes:

[0104] Step 110, when processing the current read transaction, determine the virtual identifier of the current read transaction according to the most recent transaction identifier of the current read transaction and the first preset identifier generation rule, wherein the virtual identifier of the current read transaction includes the most recent write transaction identifier and sub-identifier of the current read transaction;

[0105] Step 120, scan each write transaction record in the transaction log, and determine whether the write transaction record is visible to the current read transaction based on the write transaction identifier in the write transaction record, the virtual identifier of the current read transaction, and the isolation level of the current read transaction. If the write transaction record is visible to the current read transaction, read the data block corresponding to the write transaction record from the database.

[0106] When processing a read transaction, this embodiment can enable the read transaction to read relevant data by assigning a virtual identifier to the read transaction, and when recovering data in the event of a database failure, no read transaction record is generated for the read transaction, thereby improving data recovery efficiency.

[0107] In detail, the read transaction described in this article is only used to read data in the database and will not modify the data in the database. The current read transaction refers to the transaction being processed. When multiple threads process transactions at the same time, the current read transaction may include multiple transactions. If the most recent transaction of the current read transaction is a read transaction, the most recent transaction identifier of the current read transaction refers to the virtual identifier of the most recent historical read transaction. If the most recent transaction of the current read transaction is a write transaction, the most recent transaction identifier of the current read transaction refers to the identifier of the most recent historical write transaction. A historical write transaction refers to a write transaction that has been processed.

[0108] The virtual identifier of the read transaction is determined according to the first preset identifier generation rule. The first preset identifier generation rule makes the virtual identifier of the current read transaction include the most recent write transaction identifier and sub-identifier of the current read transaction. The specific generation logic of the sub-identifier can be set according to the needs, such as the sub-identifier value increases one by one, or the number of sub-identifier bits increases one by one. For example, if the most recent historical write transaction identifier is 001 and the virtual identifier of the historical read transaction is 0011, then the virtual identifier of the current read transaction is 0012. In specific implementation, the virtual identifier of the read transaction also includes: the read transaction process number and the sequence number and other contents.

[0109] In some implementations, the write transaction record includes at least one tuple record, such as Figure 2 As shown, each tuple record includes at least a write transaction identifier 210, a row identifier 220, all column data 230 and a mark bit 240;

[0110] The write transaction identifier 210 is used to uniquely locate the write transaction. The write transaction identifier consists of numbers and is numbered in chronological order (for example, each write transaction identifier differs by 1), that is, the later write transaction identifier is greater than the earlier write transaction identifier.

[0111] The row identifier 220 is used to determine the row position where the write-related data is located. The row identifier can also be used as the number of the tuple.

[0112] All column data 230 refers to the values ​​of all columns corresponding to the row identifier 220 , such as an update operation, all column data corresponding to the updated row, and such as a delete operation, all column data corresponding to the deleted row.

[0113] Mark bit 240 is used to identify the write operation type. When mark bit 240 indicates valid (for example, False), the corresponding operation type is an update operation or a new operation; when mark bit 240 indicates invalid (for example, True), the corresponding operation type is a delete operation.

[0114] In a specific embodiment, tuple records are implemented in a redo log format, as shown in Fig.12 As shown, it includes redo log header information, redo log data, multiple backup block header information and backup block data. Among them, the redo log header data records a cyclic redundancy check code, a previous log record pointer, a transaction identifier, a total length of the log record, a mark bit, a total length of the resource manager data, etc. The redo log data includes a next log record pointer, a starting position of the resource manager data, and a buffer. The backup block header information includes a row identifier, and the backup block data includes column data corresponding to the row identifier.

[0115] This embodiment can record multiple modification records of the same row identifier. Each modification will record a tuple record. In order to facilitate the search for the latest modified data corresponding to the row identifier, the tuple records of the same row identifier are linked by pointers. The version number of the tuple can also be recorded in each tuple. The version number can reflect the number of modifications corresponding to the row identifier data.

[0116] Specifically, Figure 3 As shown, write transaction 001 includes four new operations, and the inserted key values ​​are (1, 'AAA'), (2, 'BBB'), (3, 'CCC'), and (4, 'DDD'). Since they are newly inserted tuples, the mark bits of these four tuples are all False, indicating that they are valid tuples, which are the first versions of tuples 1, 2, 3, and 4 respectively.

[0117] Write transaction 002 includes two update operations, which updates the value of key 2 from 'BBB' to 'EEE', and updates the value of key 3 from 'CCC' to 'FFF'. Since it is an update operation, new tuples (2, 'EEE') and (3, 'FFF') are added to tuples 2 and 3 respectively, and their deletion marks are set to False, forming the second version of tuples 2 and 3.

[0118] Transaction 003 includes a delete operation, which deletes the tuple with key 3. Therefore, a new tuple (3, 'FFF') is added and its delete flag bit is set to True, indicating that the record is deleted. At this time, the third version of tuple 3 is formed.

[0119] In other implementations, in order to reduce the size of the write transaction record and further reduce the size of the transaction log, the write transaction record includes at least one tuple record, such as Figure 4 As shown, each tuple records: a write transaction identifier 310 , a row identifier 320 , difference data 330 , and a mark bit 340 , wherein the difference data 330 includes a copy item 331 , a delete item 332 , and / or a newly added item 333 .

[0120] The write transaction identifier 310 is used to uniquely locate the write transaction. The write transaction identifier consists of numbers and is numbered in chronological order (for example, each write transaction identifier differs by 1), that is, the later write transaction identifier is greater than the earlier write transaction identifier.

[0121] The row identifier 320 is used to determine the row position where the write-related data is located. The row identifier can also be used as the number of the tuple.

[0122] The flag bit 340 is used to identify the type of write operation. When the flag bit indicates valid, the corresponding operation type is an update operation or a new operation; when the flag bit indicates invalid, the corresponding operation type is a delete operation.

[0123] The copy item 331 is used to record the bytes occupied by unchanged data, the delete item 332 is used to record the bytes occupied by deleted data, and the new item 333 is used to record the modified data. The copy item 331, the delete item 332 and / or the new item 333 are arranged in the order of all columns corresponding to the row identifier.

[0124] For example, the corresponding column data in a data table include: c1 DATE, c2 INT, c3 VARCHAR(), c4 TIMESTAMP, c5 INT, c6 INT. Among them, c1 occupies 8 bytes, c2 occupies 4 bytes, c3 occupies 8 bytes, c4 occupies 8 bytes, c5 occupies 4 bytes, and c6 occupies 4 bytes.

[0125] An update operation is performed on a row of data in the data table ('2021-09-01', 2, 'abcdefghij', '2021-09-01 17:35:20', 2, 2). Specifically, c5 is updated to 5. In the prior art, the modification information recorded in the transaction log is: '2021-09-01', 2, 'abcdefghij', '2021-09-01 17:35:20', 5, 2. In this article, the modification information recorded in the transaction log is: COPY 28, IGNORE 4, ADD 4, COPY 4.

[0126] This embodiment records the difference data in the tuple record and identifies the unmodified data through identification (COPY), which can greatly reduce the number of bytes that need to be written to the tuple record. The more columns a table has, the more obvious the compression is.

[0127] In one embodiment of this invention, Figure 5 As shown in the figure, the database transaction management method also includes:

[0128] Step 510, when processing the current write transaction, determine the current write transaction identifier according to the most recent historical write transaction identifier of the current write transaction and a second preset identifier generation rule;

[0129] Step 520, writing the data page related to the current write transaction into the data buffer of the memory;

[0130] Step 530, modifying the data page in the data buffer according to the current write transaction, and refreshing the modified data page to the data area of ​​the database by the first write data service;

[0131] Step 540: construct a write transaction record in the transaction log buffer of the memory according to the current write transaction identifier and the modified data page, and the second write data service refreshes the write transaction record to the transaction log of the database.

[0132] In detail, this embodiment can be implemented with Figure 1 The illustrated embodiments are executed synchronously, and the order of execution is not limited herein.

[0133] When step 510 is implemented, the historical write transaction identifier most recent to the current write transaction is updated according to the second preset identifier generation rule, and the updated value is greater than the historical write transaction identifier most recent to the current write transaction, and the updated value is used as the current write transaction identifier. The second preset identifier generation rule is, for example, an ascending function (for example, increasing by 1 each time), and the historical write transaction identifier most recent to the current write transaction is input into the ascending function to obtain the current write transaction identifier.

[0134] In order to facilitate the determination of the start and end processing of the write transaction when scanning the transaction log, after the above step 510 determines the current write transaction identifier, a start transaction record will be generated in the transaction log buffer in the memory, and the third write data service will start the transaction record (such as Figure 6 After the transaction record is written in step 540 and refreshed to the transaction log of the database, a commit transaction record is generated in the transaction log buffer of the memory, and the fourth write data service writes the commit transaction record (as shown in FIG. Figure 6 as shown) to the database's transaction log.

[0135] When step 520 is implemented, if it is an update or delete operation, the relevant data page is obtained from the database according to the write transaction. If it is a new operation, an empty data page is constructed according to the write transaction. Considering the concurrent processing of transactions, in order to avoid operations, step 520 writes the data page related to the current write transaction to the data buffer in the memory and then adds an exclusive lock to it. The exclusive lock can restrict other transactions from writing to the data page in the memory.

[0136] When step 530 is implemented, the updated data, deleted data, or newly added data is stored in the data page in the buffer. The first data writing service may also be referred to as a bgwriter service. In order to facilitate the first data writing service to determine which data pages should be refreshed to the transaction log of the database, after modifying the data page in the data buffer, the modified data page is marked as a dirty page, and the first data writing service refreshes the dirty page to the data area of ​​the database.

[0137] When step 540 is implemented, Figure 2 or Figure 4 The format shown constructs a tuple record in a write transaction record in memory. The second write data service may be referred to as a WALwriter service. After constructing the write transaction record, the exclusive lock is released.

[0138] The first, second, third, and fourth data writing services described herein may be the same process or different processes for implementing the same function, and this document does not limit this.

[0139] In one embodiment of the present invention, the first preset identifier generation rule includes a first identifier generation sub-rule and a second identifier generation sub-rule. The above step 110 determines the virtual identifier of the current read transaction according to the most recent transaction identifier of the current read transaction and the first preset identifier generation rule, including:

[0140] If the most recent transaction of the current read transaction is a write transaction, a sub-rule is generated according to the most recent historical write transaction identifier of the current read transaction and the first identifier to determine the virtual identifier of the current read transaction;

[0141] If the most recent transaction of the current read transaction is a read transaction, a sub-rule is generated according to the most recent historical read transaction identifier and the second identifier of the current read transaction to determine the virtual identifier of the current write transaction.

[0142] In detail, the first identifier generation sub-rule is used to generate a sub-identifier and concatenate the write transaction identifier and sub-identifier of the current read transaction, and use the concatenated identifier as the virtual identifier of the current read transaction. For example, the write transaction identifier of the current read transaction is 003, and the sub-identifier value is 01, then the virtual identifier of the current read transaction is 00301. There are many ways to generate sub-identifiers, which are not limited in this document.

[0143] The second identifier generation sub-rule is used to update the historical read transaction identifier most recent to the current read transaction, keep the write transaction identifier in the historical read transaction identifier most recent to the current read transaction unchanged during the update, adjust the sub-identifier in the historical read transaction identifier most recent to the current read transaction, and use the updated identifier as the virtual identifier of the current read transaction. For example, the historical read transaction identifier most recent to the current read transaction is 11102 (where 111 is the write transaction identifier and 02 is the sub-identifier), and the identifier obtained after the sub-identifier is updated is 11103. When adjusting the sub-identifier, it can be implemented in a one-to-one increment, or in a one-to-one increment by the number of sub-identifier digits. This article does not limit the specific adjustment method.

[0144] In one embodiment of this invention, Figure 7 As shown, the above step 120 determines whether the write transaction record is visible to the current read transaction according to the write transaction identifier in the write transaction record, the virtual identifier of the current read transaction, and the isolation level of the current read transaction, including:

[0145] Step 710, determining whether the write transaction identifier in the write transaction record is smaller than the virtual identifier of the current read transaction;

[0146] Step 720, determining whether the tuple record in the write transaction record is the latest record corresponding to the row identifier; if the determination result is yes, executing step 730; if the determination result is no, determining that the tuple record in the write transaction record is not visible to the read transaction;

[0147] Step 730: Determine whether the tuple record in the write transaction record is visible to the current read transaction according to the isolation level of the current read transaction.

[0148] In detail, since the write transaction identifier and the virtual identifier of the read transaction can determine the time sequence of transaction processing, and the identifier generated later is greater than the identifier generated earlier, the write transaction record generated before the current read transaction can be determined through step 710.

[0149] Usually, each row of data in a data table corresponds to multiple tuple records, and only the latest tuple record is the final valid record. The latest tuple record corresponding to each row of data can be found through step 720.

[0150] Different transaction configurations have different isolation levels and corresponding visible ranges. The isolation level of a transaction can be predetermined and is not limited herein. Step 730 can filter out tuple records visible to the current read transaction, and then read data blocks corresponding to the visible tuple records from the database.

[0151] by Figure 3 For example, without considering the isolation level, suppose there is a read transaction 0031 after write transaction 003. Since read transaction 0031 is later than write transaction 0003, according to the principle of displaying the highest version value of each tuple, read transaction 0031 can see the first version of tuples 1 and 4, the second version of tuple 2, and the third version of tuple 3. Suppose there is a read transaction 0021 after write transaction 002. According to the principle of displaying the highest version value of each tuple, read transaction 0021 can see the first version of tuples 1 and 4, and the second version of tuples 2 and 3.

[0152] After the above database transaction management method is used to manage transactions, when a database failure occurs, the transaction recovery process includes: forward scanning the write transaction records in the transaction log, and performing redo operations on the committed transactions and rolled back transactions.

[0153] This article records all kinds of write operations, so when recovering a transaction, you only need to scan the transaction records forward to complete the transaction recovery.

[0154] Specifically, if the write transaction record at least includes: a write transaction identifier, a row identifier, difference data, and a mark bit, wherein the difference data includes a copy item, a deletion item, and / or a new item, such as Figure 8 As shown in the figure, redo operations are performed on committed transactions and rolled back transactions, including:

[0155] Step 810, for each committed transaction and each rolled back transaction, find the first tuple record corresponding to each tuple record in the committed transaction and the rolled back transaction;

[0156] Step 820, restore the modified data according to each tuple record in the committed transaction and its corresponding first tuple record;

[0157] Step 830, restore the modified data according to each tuple record in the rolled back transaction and its corresponding first tuple record;

[0158] Step 840, writing the modified data into the data area of ​​the database.

[0159] In detail, the first tuple record found in step 810 is the data inserted by the new operation, which records the full amount of data. Assume that the column vector in a table is defined as (c1 DATE, c2 INT, c3 VARCHAR(), c4TIMESTAMP, c5 INT, c6 INT), where c1 occupies 8 bytes, c2 occupies 4 bytes, c3 occupies 8 bytes, c4 occupies 8 bytes, c5 occupies 4 bytes, and c6 occupies 4 bytes. The data in the first tuple record is ('2021-09-01', 2, 'abcdefghij', '2021-09-01 17:35:20', 2, 2), and the difference data in the committed transaction is (COPY 28, IGNORE 4, 5, COPY 4). COPY28 corresponds to the data of c1, c2, c3, and c4 of the original data, and IGNORE 4 deletes c5, COPY4 corresponds to the source data c6, and based on ('2021-09-01', 2, 'abcdefghij', '2021-09-01 17:35:20', 2, 2) and (COPY 28, IGNORE 4, 5, COPY 4), the data ('2021-09-01', 2, 'abcdefghij', '2021-09-01 17:35:20', 5, 2) can be restored. Based on the same inventive concept, this article also provides a database transaction management device and a database transaction recovery device, as described in the following embodiments. Since the principles of solving problems by the database transaction management device and the database transaction recovery device are similar to those of the database transaction management method and the database transaction recovery method, the implementation of the database transaction management device and the database transaction recovery device can refer to the database transaction management method and the database transaction recovery method, and the repeated parts will not be repeated.

[0160] Specifically, Fig. 9 As shown, the database transaction management device includes:

[0161] The read transaction processing module 910 is used to determine the virtual identifier of the current read transaction according to the most recent transaction identifier of the current read transaction and the first preset identifier generation rule when processing the current read transaction, wherein the virtual identifier of the current read transaction includes the most recent write transaction identifier and sub-identifier of the current read transaction;

[0162] The scanning module 920 is used to scan each write transaction record in the transaction log, and determine whether the write transaction record is visible to the current read transaction based on the write transaction identifier in the write transaction record, the virtual identifier of the current read transaction, and the isolation level of the current read transaction; if the write transaction record is visible to the current read transaction, read the data block corresponding to the write transaction record from the database.

[0163] Further, such as Fig.10 As shown, the database transaction management device also includes:

[0164] The write transaction processing module 930 determines the current write transaction identifier according to the most recent historical write transaction identifier of the current write transaction and the second preset identifier generation rule when processing the current write transaction; writes the data page related to the current write transaction to the data buffer of the memory; modifies the data page in the data buffer according to the current write transaction, and the write data service refreshes the modified data page to the data area of ​​the database; constructs a write transaction record in the transaction log buffer of the memory according to the current write transaction identifier and the modified data page, and the write data service refreshes the write transaction record to the transaction log of the database.

[0165] This embodiment generates a write transaction identifier (i.e., an actual transaction identifier) ​​only for a write transaction, and does not generate an actual transaction identifier for a read transaction, but generates a virtual identifier, and the read transaction does not generate a log record, thereby reducing the length of the log scanned during recovery, thereby greatly improving the recovery efficiency and reducing the problem of wraparound due to insufficient transaction identifiers.

[0166] like Fig.11 As shown, the database transaction recovery device includes:

[0167] Scanning module 1110, used to forward scan the write transaction records in the transaction log when a system failure occurs, to determine the committed transactions and the rolled back transactions;

[0168] The redo module 1120 is used to perform redo operations on submitted transactions and rolled back transactions, wherein the tuples in the write transaction record are written in Figure 2 and Figure 4 The schematic diagram described is recorded.

[0169] In one embodiment of the present invention, a computer device is also provided for executing the steps of any of the above method embodiments. Fig.13As shown, the computer device 1302 may include one or more processors 1304, such as one or more central processing units (CPUs), each of which may implement one or more hardware threads. The computer device 1302 may also include any memory 1306, which is used to store any kind of information such as code, settings, data, etc. Non-limitingly, for example, the memory 1306 may include any one or more combinations of the following: any type of RAM, any type of ROM, flash memory device, hard disk, optical disk, etc. More generally, any memory may use any technology to store information. Further, any memory may provide volatile or non-volatile retention of information. Further, any memory may represent a fixed or removable component of the computer device 1302. In one case, when the processor 1304 executes an associated instruction stored in any memory or a combination of memories, the computer device 1302 may perform any operation of the associated instruction. The computer device 1302 also includes one or more drive mechanisms 1308 for interacting with any memory, such as a hard disk drive mechanism, an optical disk drive mechanism, etc.

[0170] The computer device 1302 may also include an input / output module 1310 (I / O) for receiving various inputs (via input devices 1312) and for providing various outputs (via output devices 1314). A specific output mechanism may include a presentation device 1316 and an associated graphical user interface 1318 (GUI). In other embodiments, the input / output module 1310 (I / O), input device 1312, and output device 1314 may not be included, and the computer device 1302 may be used as a computer device in a network. The computer device 1302 may also include one or more network interfaces 1320 for exchanging data with other devices via one or more communication links 1322. One or more communication buses 1324 couple the components described above together.

[0171] The communication link 1322 may be implemented in any manner, for example, through a local area network, a wide area network (e.g., the Internet), a point-to-point connection, etc., or any combination thereof. The communication link 1322 may include any combination of hardwired links, wireless links, routers, gateway functions, name servers, etc., governed by any protocol or combination of protocols.

[0172] Corresponds to Figure 1 , Figure 5 , Figure 7 , Figure 8 The method in this article also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above method are executed.

[0173] The embodiment of the present invention also provides a computer readable program, wherein when a processor in a computer executes the program, the program causes the processor to execute the following Figure 1 , Figure 5 , Figure 7 , Figure 8 The method shown.

[0174] It should be understood that in the various embodiments of this document, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this document.

[0175] It should also be understood that in the embodiments of this article, the term "and / or" is only a description of the association relationship of the associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0176] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this article.

[0177] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0178] In the several embodiments provided herein, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, or can be electrical, mechanical or other forms of connection.

[0179] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of this article.

[0180] In addition, each functional unit in each embodiment of this invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit may be implemented in the form of hardware or in the form of software functional unit.

[0181] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this article is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of this article. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0182] Specific embodiments are used in this article to illustrate the principles and implementation methods of this article. The description of the above embodiments is only used to help understand the methods and core ideas of this article. At the same time, for general technicians in this field, according to the ideas of this article, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on this article.

Claims

1. A database transaction management method, characterized in that: include: When processing the current read transaction, a virtual identifier of the current read transaction is determined according to the most recent historical transaction identifier of the current read transaction and a first preset identifier generation rule, wherein the virtual identifier of the current read transaction includes the most recent write transaction identifier and sub-identifier of the current read transaction; Scan each write transaction record in the transaction log, and determine whether the write transaction record is visible to the current read transaction according to the write transaction identifier in the write transaction record, the virtual identifier of the current read transaction, and the isolation level of the current read transaction; if the write transaction record is visible to the current read transaction, read the data block corresponding to the write transaction record from the database; When processing the current write transaction, determining the current write transaction identifier according to the most recent historical write transaction identifier of the current write transaction and a second preset identifier generation rule; Writing the data page related to the current write transaction into the data buffer of the memory; According to the current write transaction, the data page in the data buffer is modified, and the first write data service refreshes the modified data page to the data area of ​​the database; According to the current write transaction identifier and the modified data page, a write transaction record is constructed in the transaction log buffer of the memory, and the second write data service refreshes the write transaction record to the transaction log of the database.

2. The database transaction management method according to claim 1, characterized in that: After determining the current write transaction identifier, generating a start transaction record in a transaction log buffer in the memory, and refreshing the start transaction record to the transaction log of the database by the third write data service; After the write transaction record is refreshed into the transaction log of the database, a commit transaction record is generated in the transaction log buffer of the memory, and the fourth write data service refreshes the commit transaction record into the transaction log of the database.

3. The database transaction management method according to claim 1, characterized in that: The write transaction record includes at least one tuple record, and each tuple record includes at least a write transaction identifier, a row identifier, all column data, and a mark bit; The flag bit is used to identify the write operation type. When the flag bit indicates that the write operation type is valid, the corresponding operation type is an update operation or a new operation. When the flag bit indicates invalid, the corresponding operation type is a delete operation.

4. The database transaction management method according to claim 1, characterized in that: The write transaction record includes at least one tuple record, each tuple record includes: a write transaction identifier, a row identifier, difference data, and a mark bit, wherein the difference data includes a copy item, a deleted item, and / or a newly added item; The flag bit is used to identify the write operation type. When the flag bit indicates valid, the corresponding operation type is an update operation or a new operation; when the flag bit indicates invalid, the corresponding operation type is a delete operation. The copy item is used to record the bytes occupied by unchanged data, the delete item is used to record the bytes occupied by deleted data, and the new item is used to record modified data; The copied items, deleted items and / or newly added items are arranged in the order of all columns corresponding to the row identifiers.

5. The database transaction management method according to claim 3 or 4, characterized in that: Tuple records with the same row identifier are linked by pointers.

6. The database transaction management method according to claim 1, characterized in that: After writing the data page related to the current write transaction into the data buffer of the memory, an exclusive lock is added to the data page; After constructing the write transaction record, the exclusive lock is released.

7. The database transaction management method according to claim 1, characterized in that: The first preset identifier generation rule includes a first identifier generation sub-rule and a second identifier generation sub-rule; Determining the virtual identifier of the current read transaction according to the most recent historical transaction identifier of the current read transaction and a first preset identifier generation rule includes: If the most recent historical transaction of the current read transaction is a write transaction, generating a sub-rule according to the most recent historical write transaction identifier of the current read transaction and the first identifier to determine the virtual identifier of the current read transaction; If the most recent historical transaction of the current read transaction is a read transaction, a sub-rule is generated according to the most recent historical read transaction identifier of the current read transaction and a second identifier to determine a virtual identifier of the current read transaction.

8. The database transaction management method according to claim 7, characterized in that: Determining the virtual identifier of the current read transaction according to the historical write transaction identifier most recent to the current read transaction and the first identifier generating sub-rule includes: Generate a sub-rule using the first identifier, generate a sub-identifier and concatenate the most recent historical write transaction identifier and the sub-identifier of the current read transaction; The concatenated identifier is used as the virtual identifier of the current read transaction.

9. The database transaction management method according to claim 7, characterized in that: Determining the virtual identifier of the current read transaction according to the historical read transaction identifier most recent to the current read transaction and the second identifier generating sub-rule includes: The second identifier is used to generate a sub-rule, and the most recent historical read transaction identifier of the current read transaction is updated. During the update, the write transaction identifier in the most recent historical read transaction identifier of the current read transaction is kept unchanged, and the sub-identifier in the most recent historical read transaction identifier of the current read transaction is adjusted; The updated identifier is used as the virtual identifier of the current read transaction.

10. The database transaction management method according to claim 2, characterized in that: Determining whether the write transaction record is visible to the current read transaction according to the write transaction identifier in the write transaction record, the virtual identifier of the current read transaction, and the isolation level of the current read transaction includes: Determine whether the write transaction identifier in the write transaction record is less than the virtual identifier of the current read transaction; Determine whether the tuple record in the write transaction record is the latest record corresponding to the row identifier; If the judgment results are all yes, determining whether the tuple record in the write transaction record is visible to the current read transaction according to the isolation level of the current read transaction; If the judgment result is no, it is determined that the tuple record in the write transaction record is not visible to the read transaction.

11. A database transaction recovery method, characterized in that: The method of claim 1 is used to record transaction logs, and the transaction recovery process when a database failure occurs includes: Scan the write transaction records in the transaction log forward and perform redo operations on committed transactions and rolled back transactions.

12. The database transaction recovery method according to claim 11, characterized in that: If the write transaction record at least includes: a write transaction identifier, a row identifier, difference data, and a mark bit, wherein the difference data includes a copy item, a deletion item, and / or a new item, performing a redo operation on a committed transaction and a rolled back transaction includes: For each committed transaction and each rolled back transaction, find the first tuple record corresponding to each tuple record in the committed transaction and the rolled back transaction; Restore the modified data according to each tuple record in the committed transaction and its corresponding first tuple record; Restore the modified data according to each tuple record in the rolled back transaction and its corresponding first tuple record; Write the modified data into the data area of ​​the database.

13. A database transaction management device, characterized in that: include: a read transaction processing module, configured to determine, when processing a current read transaction, a virtual identifier of the current read transaction according to a recent historical transaction identifier of the current read transaction and a first preset identifier generation rule, wherein the virtual identifier of the current read transaction includes a recent write transaction identifier and a sub-identifier of the current read transaction; a scanning module, configured to scan each write transaction record in the transaction log, and determine whether the write transaction record is visible to the current read transaction according to the write transaction identifier in the write transaction record, the virtual identifier of the current read transaction, and the isolation level of the current read transaction; if the write transaction record is visible to the current read transaction, then read the data block corresponding to the write transaction record from the database; The write transaction identifier determination process includes: when processing the current write transaction, determining the current write transaction identifier according to the most recent historical write transaction identifier of the current write transaction and a second preset identifier generation rule; Writing the data page related to the current write transaction into the data buffer of the memory; According to the current write transaction, the data page in the data buffer is modified, and the first write data service refreshes the modified data page to the data area of ​​the database; According to the current write transaction identifier and the modified data page, a write transaction record is constructed in the transaction log buffer of the memory, and the second write data service refreshes the write transaction record to the transaction log of the database.

14. A database transaction recovery device, characterized in that: include: A scanning module is used to forward scan the write transaction records in the transaction log when a system failure occurs, and perform redo operations on submitted transactions and rolled back transactions, wherein the write transaction records are recorded using the method described in claim 1.

15. A computer device comprising a memory, a processor, and a computer program stored in the memory, characterized in that: When the computer program is executed by the processor, the computer program executes the instructions of the method according to any one of claims 1 to 10.

16. A computer storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor of a computer device, the computer program executes the instructions of the method according to any one of claims 1 to 10.

17. A computer readable program, characterized in that When a processor in a computer executes the program, the program executes the instructions of the method according to any one of claims 1 to 10.

Images