Database log recording method and system

Abstract

The present invention relates to the field of electronic digital data processing technology, and in particular to a database log recording method and system, the method comprising the following steps: inputting a response update instruction, and determining the update column type, if the update column type includes a variable-length column, then adding a variable-length column mark to the update column and transmitting it to an executor, if the update column type does not include a variable-length column, then adding a fixed-length column mark to the update column and transmitting it to the executor; if the update column is a variable-length column mark, then the executor directly saves the update column to a rear mirror image, if the update column is a fixed-length column mark, then the update column is compared with the pre-update data in the front mirror image, and the update column data is compressed using a differential compression method and then saved to the rear mirror image. The method and system provided by the present invention achieve a large amount of compression of the storage space required for the logical log volume, and improve the speed of data recovery or playback.

Description

Technical Field

[0001] The present invention relates to the technical field of electronic digital data processing, and in particular to a database log recording method and system. Background Art

[0002] Database systems usually need to ensure the availability and reliability of data, which requires backup and recovery operations on the database system. In the related art, logical logs are generally used to record the operation transactions of the system, and the operation transactions here include but are not limited to insertion, deletion, and update. However, the logical log generally only records the transaction operation process. When performing data recovery, when referring to the logical log, if a process record error occurs, it is easy to cause data recovery failure. To solve this problem, the current status of the log recording device is that when receiving a response and an update instruction, a logical log is generated. The logical log includes a front image and a rear image. The front image and the rear image both include all data storage of the rows involved in the table before and after the update instruction. After the update instruction is completed, the storage operation in the rear image is still performed for the columns that have not been updated, so that the log recording device has a large amount of redundant data storage, which seriously affects the speed of data recovery or playback. Summary of the invention

[0003] The technical problem to be solved by the present invention is to provide a database log recording method and system, which changes the logical log structure and divides the updated column into variable-length columns and fixed-length columns according to the data type of the updated column. For the updated column containing the variable-length column, the updated column is directly saved to the back image, and for the fixed-length column not containing the variable-length column, it is saved to the back image in a compressed manner, thereby achieving a large amount of compression of the storage space required for the logical log volume and improving the speed of data recovery or playback.

[0004] The present invention is achieved through the following technical solutions:

[0005] A database log recording method comprises the following steps:

[0006] S1: Input a response update instruction and generate a logical log. The structure of the logical log includes a front image and a back image. The front image is used to store data before the update, and the back image is used to store data after the update.

[0007] S2: Determine the update column type involved in the update instruction. If the update column type includes a variable-length column, add a variable-length column mark bit to the update column and transmit it to the executor. If the update column type does not include a variable-length column, add a fixed-length column mark bit to the update column and transmit it to the executor.

[0008] S3: The executor processes according to the mark bit of the update column. If the update column is a variable-length column mark bit, the update column is directly saved to the rear image. If the update column is a fixed-length column mark bit, the update column is compared with the data before the update in the front image, and the update column data is compressed using the differential compression method, and the compressed data is saved to the rear image.

[0009] Furthermore, a database log recording method also includes recovery or playback of the updated column.

[0010] Furthermore, the method for recovering or replaying the update column is as follows: input a data recovery or playback instruction; if the update column type includes a variable-length column, the executor directly retrieves the update column data of the rear image and displays it; if the update column type does not include a variable-length column, the executor decompresses the compressed data saved in the rear image, replaces the fields before the update with the fields after the update, forms the update column data and displays it.

[0011] Preferably, in step S2, the mark bit of the variable-length column is set to 0, and the mark bit of the fixed-length column is set to 1.

[0012] Preferably, in step S3, when the update column is marked as a fixed-length column, the following method is used to compress the update column data: compare the update column with the data before the update in the front image, compress the update column into compressed data including the starting position of the fixed field before the changing field, the fixed field length before the changing field, the changing field, the starting position of the fixed field after the changing field and the fixed field length after the changing field, and add a copy mark before the starting position of the fixed field before the changing field and before the starting position of the fixed field after the changing field, and then save the compressed data to the rear image.

[0013] Preferably, in step S3, when the update column is marked as a fixed-length column, the following method is used to compress the update column data: compare the update column with the data before the update in the front image, compress the update column to include the number of modified blocks, the starting position of the modified block, the modified block field before modification, and the modified block field after modification, and then save the compressed data in the rear image.

[0014] A database log recording system, used to execute any of the above database log recording methods, comprising a response module and an executor;

[0015] The response module is used to respond to the update instruction and generate a logical log, wherein the logical log structure includes a front image and a rear image, wherein the front image is used to store the data before the update, and the rear image is used to store the data after the update; the update column type involved in the update instruction is determined, and if the update column type includes a variable-length column, the update column is added with a variable-length column mark bit and then transmitted to the executor; if the update column type does not include a variable-length column, the update column is added with a fixed-length column mark bit and then transmitted to the executor;

[0016] The executor is used to process data according to the mark bit of the update column. If the update column is a variable-length column mark bit, the update column is directly saved in the rear image. If the update column is a fixed-length column mark bit, the update column is compared with the data before the update in the front image, and the update column data is compressed using a differential compression method, and the compressed data is saved in the rear image.

[0017] Furthermore, a database logging system also includes a display, and the executor is also used to execute data recovery or playback instructions. If the update column type includes variable-length columns, the executor directly retrieves the update column data of the rear image and displays it. If the update column type does not include variable-length columns, the executor decompresses the compressed data saved in the rear image, replaces the fields before the update with the fields after the update, forms the update column data and displays it by the display.

[0018] Beneficial effects of the invention:

[0019] The present invention changes the logical log structure and divides the updated column into variable-length columns and fixed-length columns according to the data type of the updated column. For the updated column containing the variable-length column, the updated column is directly saved to the back image, and for the fixed-length column not containing the variable-length column, the updated column is saved to the back image in a compressed manner, thereby achieving a large amount of compression of the storage space required for the logical log volume and improving the speed of data recovery or playback. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a schematic diagram of the overall process of the present invention. DETAILED DESCRIPTION

[0021] A database log recording method, the overall flow diagram is as follows Figure 1 As shown, the specific steps include:

[0022] S1: Input a response update instruction and generate a logical log. The structure of the logical log includes a front image and a rear image. The front image is used to save the data before the update, and the rear image is used to save the data after the update. The front image in the logical log here is the data before the update, and the rear image is the data after the update. By changing the logical log structure, the data before the update and the data after the update are directly saved respectively, which ensures the consistency of the data. When restoring or replaying the data, there is no need to read the intermediate operation records, which improves the data recovery speed and improves the performance and availability of the database.

[0023] S2: Determine the update column type involved in the update instruction. If the update column type includes a variable-length column, a variable-length column mark bit is added to the update column and then transmitted to the executor. If the update column type does not include a variable-length column, a fixed-length column mark bit is added to the update column and then transmitted to the executor.

[0024] The specific variable-length column flag bit can be set to 0, and the fixed-length column flag bit can be set to 1. Using a flag bit of 0 or 1 to distinguish whether the updated column type is a variable-length column or a fixed-length column is simple, practical, and easy to identify.

[0025] S3: The executor processes according to the mark bit of the update column. If the update column is a variable-length column mark bit, the update column is directly saved to the rear image. If the update column is a fixed-length column mark bit, the update column is compared with the data before the update in the front image, and the update column data is compressed using the differential compression method, and the compressed data is saved to the rear image.

[0026] If the update column is a variable-length column mark, the executor directly saves the update column to the back image. If the update column is a fixed-length column mark, the executor compares the update column with the data before the update in the front image, and uses the differential compression method to compress the update column data, and finally only saves the compressed data in the back image. This can reduce the storage of a large amount of redundant data in the back image, which is conducive to improving the speed of data recovery or playback and improving the performance of the database.

[0027] Specifically, when the update column is marked as a fixed-length column, the following method can be used to compress the update column data: compare the update column with the data before the update in the front image, compress the update column into compressed data including the starting position of the fixed field before the changing field, the fixed field length before the changing field, the changing field, the starting position of the fixed field after the changing field, and the fixed field length after the changing field, and add a copy mark before the starting position of the fixed field before the changing field and before the starting position of the fixed field after the changing field, and then save the compressed data in the back image.

[0028] For example: Suppose the data stored in a table includes the first field, the second field and the third field, where the first field and the second field are fixed-length column data. The first field only needs to be allocated 3 field lengths during the storage process, such as 3 is recorded as 003, 34 is recorded as 034, and 101 is recorded as 101. The second field is 18 fields, and the third field is a variable-length column data. The third field of each column records different information, and its storage length is different.

[0029] Assume that the data before update in the table is (23, xxxxxx2000xxxxxxxx, xxxxxxxxxx). When an update command is input, the database statement engine layer of the response module will judge its type.

[0030] Assume that the update instruction is to modify only the following information: the first field changes 23 to 33, then the executor uses a compression algorithm to save only 33 C 3 28 in the rear image, where 33 indicates the changed field, C means copy, 3 indicates the starting field position of the copy, and 28 indicates the length of the copied field.

[0031] Assume that the second field needs to be modified and the 2000 in the second field needs to be changed to 1990. The executor uses a compression algorithm to save only C 1 8 1990 C 13 18 in the rear image. C means copy, 1 indicates the starting field position of the copy, 8 indicates the length of the field to be copied, 1990 is the changed field, 13 indicates that the copy starts from the 13th bit after the changed field, and the length of the copied field is 18.

[0032] Preferably, in step S3, when the update column is marked as a fixed-length column, the following method is used to compress the update column data: compare the update column with the data before the update in the front image, compress the update column to include the number of modified blocks, the starting position of the modified block, the modified block field before modification, and the modified block field after modification, and then save the compressed data in the rear image.

[0033] For example: Still taking the above table as an example, assuming that the data before update in the table is (23, xxxxxx2000xxxxxxxx, xxxxxxxxxx), to modify the first field to 33, the executor uses a compression algorithm to only save 1 1 23 33 in the rear image, where the first 1 indicates that the number of modified blocks is 1, the second 1 indicates the starting position of the first modified block, 23 indicates the field before update, and 33 indicates the field after update.

[0034] Assuming that the second field needs to be modified and 2000 in the second field needs to be changed to 1990, the executor uses a compression algorithm to save only 1 9 2000 1990 in the rear image, where 1 indicates the number of modified blocks is 1, 9 indicates the starting position of the modified block, 2000 is the field before the update, and 1990 indicates the field after the update.

[0035] If the first field and the second field are to be modified together, the executor uses a compression algorithm to only save 22 1 23 33 4 9 2000 1990 in the rear image, where the first 2 indicates that the number of modification blocks is 2, the second 2 indicates the length of the first modification block, 1 indicates the starting position of the first modification block, 23 indicates the field before the first modification block is updated, 33 indicates the field after the first modification block is updated, 4 indicates the length of the second modification block, 9 indicates the starting position of the second modification block, 2000 indicates the field before the second modification block is updated, and 1990 indicates the field after the second modification block is updated.

[0036] If the update column involves modification of the third field, it contains a variable-length column, and the update column data can be directly saved in the rear image.

[0037] From the above, it can be seen that through a database log recording method of the present invention, according to the data type of the update column, it is divided into variable-length columns and fixed-length columns. For the update column containing variable-length columns, the update column is directly saved to the back image, and for the fixed-length column not containing variable-length columns, it is saved to the back image in a compressed manner. A large amount of compression of the storage space required for the logical log volume can be achieved, thereby improving the speed of data recovery or playback.

[0038] Furthermore, a database log recording method also includes recovery or playback of the updated column.

[0039] Furthermore, the method for recovering or replaying the update column is as follows: input a data recovery or playback instruction; if the update column type includes a variable-length column, the executor directly retrieves the update column data of the rear image and displays it; if the update column type does not include a variable-length column, the executor decompresses the compressed data saved in the rear image, replaces the fields before the update with the fields after the update, forms the update column data and displays it.

[0040] When restoring or replaying the updated column, it is only necessary to decompress it according to the specific compression method used. The method is simple, convenient and fast.

[0041] A database log recording system, used to execute any of the above database log recording methods, comprising a response module and an executor;

[0042] The response module is used to respond to the update instruction and generate a logical log, wherein the logical log structure includes a front image and a rear image, wherein the front image is used to store the data before the update, and the rear image is used to store the data after the update; and the update column type involved in the update instruction is determined, and if the update column type is a variable-length column, the update column is added with a variable-length column mark bit and then transmitted to the executor; if the update column type is a fixed-length column, the update column is added with a fixed-length column mark bit and then transmitted to the executor;

[0043] The executor is used to process according to the mark bit of the update column. If the update column is a variable-length column mark bit, the update column is directly saved in the rear image. If the update column is a fixed-length column mark bit, the update column is compared with the data before the update in the front image, and the update column is converted into compressed data including the starting position of the fixed field before the changing field, the length of the fixed field before the changing field, the changing field, the starting position of the fixed field after the changing field and the length of the fixed field after the changing field, and the compressed data is saved in the rear image.

[0044] Furthermore, a database logging system also includes a display, and the executor is also used to execute data recovery or playback instructions. If the update column type includes variable-length columns, the executor directly retrieves the update column data of the rear image and displays it. If the update column type does not include variable-length columns, the executor decompresses the compressed data saved in the rear image, replaces the fields before the update with the fields after the update, forms the update column data and displays it by the display.

[0045] In summary, the present invention provides a database log recording method and system, which directly saves the updated columns containing variable-length columns to the back image, and saves the fixed-length columns that do not contain variable-length columns to the back image in a compressed manner, thereby achieving a large amount of compression of the storage space required for the logical log volume and improving the speed of data recovery or playback.

[0046] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A database log recording method, characterized in that: The steps include: S1: Input a response update instruction and generate a logical log. The structure of the logical log includes a front image and a back image. The front image is used to store data before the update, and the back image is used to store data after the update. S2: Determine the update column type involved in the update instruction. If the update column type includes a variable-length column, add a variable-length column mark bit to the update column and transmit it to the executor. If the update column type does not include a variable-length column, add a fixed-length column mark bit to the update column and transmit it to the executor. S3: The executor processes according to the flag bit of the update column. If the update column is a variable-length column flag bit, the update column is directly saved to the rear image. If the update column is a fixed-length column flag bit, the update column is compared with the data before the update in the front image, and the update column data is compressed using the differential compression method, and the compressed data is saved to the rear image. When the update column is a fixed-length column mark, the update column data is compressed by the following method: the update column is compared with the data before the update in the front image, and the update column is compressed into compressed data including the fixed field starting position before the changed field, the fixed field length before the changed field, the changed field, the fixed field starting position after the changed field, and the fixed field length after the changed field, and a copy mark is added before the fixed field starting position before the changed field and before the fixed field starting position after the changed field, and then the compressed data is saved to the back image; Alternatively, when the update column is marked as a fixed-length column, the update column data is compressed using the following method: the update column is compared with the data before the update in the front image, the update column is compressed to include the number of modified blocks, the starting position of the modified block, the modified block field before the modification, and the modified block field after the modification, and then the compressed data is saved in the rear image.

2. A database log recording method according to claim 1, characterized in that: This also includes the restoration or replay of updated columns.

3. A database log recording method according to claim 2, characterized in that: The method for restoring or replaying the update column is as follows: input the data recovery or playback instruction. If the update column type contains variable-length columns, the executor directly calls the update column data of the rear image and displays it. If the update column type does not contain variable-length columns, the executor decompresses the compressed data saved in the rear image, replaces the fields before the update with the fields after the update, forms the update column data and displays it.

4. A database log recording method according to claim 1, characterized in that: In step S2, the variable-length column flag bit is set to 0, and the fixed-length column flag bit is set to 1.

5. A database log recording system, used to execute the database log recording method according to any one of claims 1 to 4, characterized in that: including a response module and an actuator; The response module is used to respond to the update instruction and generate a logical log, wherein the logical log structure includes a front image and a rear image, wherein the front image is used to store the data before the update, and the rear image is used to store the data after the update; the update column type involved in the update instruction is determined, and if the update column type includes a variable-length column, the update column is added with a variable-length column mark bit and then transmitted to the executor; if the update column type does not include a variable-length column, the update column is added with a fixed-length column mark bit and then transmitted to the executor; The executor is used to process data according to the mark bit of the update column. If the update column is a variable-length column mark bit, the update column is directly saved in the rear image. If the update column is a fixed-length column mark bit, the update column is compared with the data before the update in the front image, and the update column data is compressed using a differential compression method, and the compressed data is saved in the rear image.

6. A database log recording system according to claim 5, characterized in that: It also includes a display, and the executor is also used to execute data recovery or playback instructions. If the update column type includes a variable-length column, the executor directly retrieves the update column data of the rear image and displays it. If the update column type does not include a variable-length column, the executor decompresses the compressed data saved in the rear image, replaces the fields before the update with the fields after the update, forms the update column data and displays it by the display.

Images