A method for monitoring data tampering based on append mode checking

By checking the repetition of primary keys based on the append mode in the database, the problems of high consumption of data tamper monitoring performance and security risks in the prior art are solved, and high security and universal data tamper detection are realized. Various database systems.

CN114491634BActive Publication Date: 2024-11-22ORANGE FRAME TECH (JIANGSU) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210104719.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-01-28
Publication Date
2024-11-22
Estimated Expiration
2042-01-28

AI Technical Summary

Technical Problem

Existing database systems have high performance consumption and data security risks when monitoring data tampering, especially DML logging is prone to secondary tampering, and logical replication and primary and standby synchronization methods require additional resources and bring data security problems.

Method used

Using an append mode-based data tamper monitoring method, data tampering is detected by calculating the primary key repeatability in the data storage mechanism, using independent files to store tagged data, and reading data block content and decompression in row or column storage database Data, form a detection table to determine the repetition of the primary key, and output data change records.

Benefits of technology

It realizes high security detection of database data tampering without increasing network transmission and resource consumption. It is suitable for various relational and distributed databases, including Postgresql, Greenplum, MySQL, etc., improving the universality of data monitoring. and security.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114491634B_ABST
    Figure CN114491634B_ABST
Patent Text Reader

Abstract

The present invention relates to a method for monitoring data tampering based on append mode inspection, and belongs to the field of information technology. The present invention includes step one: configuring data tables and attribute information that need to be monitored; step two: determining file data of a database; step three: iteratively reading data content of a corresponding data block size in a table file of the database; step four: forming a detection table; step five: determining whether there are repeated primary keys in the detection table; step six: determining and extracting data change records before and after data tampering. The present invention has strong universality and can be applied to relatively popular relational databases or distributed databases in the industry, such as Postgresql, Greenplum, MySQL, Mariadb, etc. The present method can realize data tampering detection by only reading database table files, without additional network data transmission, and its security is also relatively high.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to a data tampering monitoring method based on appending mode inspection, belonging to the field of information technology. Background Art

[0002] The data in the database has always been an important asset for the development of enterprises. The reliability, completeness and authenticity of data are the basic attributes of data records. However, in reality, due to various purposes or interests, data is often modified artificially, which in turn affects the efficiency of data use.

[0003] In order to monitor whether the database data has been modified, the log audit function has been introduced in the existing database system, which can enable DDL or DML level log records for database operations. However, due to the performance consumption of log records themselves, the log audit level of the database is often only enabled for DDL. Even if the DML function is enabled, there is still the possibility that the DML log records will be tampered with twice.

[0004] Another way to deal with this is to use similar logical replication or master-slave synchronization technology to synchronize new or modified data to other locations in a timely manner, and then perform secondary analysis at other locations to detect whether the data may have been modified. This method requires the provision of a second system library, the network bandwidth also needs to be upgraded, and it also brings pressure on the original library to divert traffic. In addition, because the data is mirrored to a third-party system, it also brings data security issues. Summary of the invention

[0005] The technical problem to be solved by the present invention is to prevent the possibility of secondary tampering of database data and make the monitoring process safer.

[0006] The data tampering monitoring method based on append mode inspection of the present invention is specifically based on the data storage mechanism of the database, and realizes the discovery of data tampering phenomenon by calculating the duplication of the class primary key caused by the data in the library table managed in append mode. The storage of the marked data of the present invention adopts the marked data to be stored in an independent file.

[0007] In the file data of the database, if the database adopts row storage, each data table corresponds to one or more heap files; if the database adopts column storage, each field corresponds to one or more segment files.

[0008] The method for monitoring data tampering based on append mode checking of the present invention specifically comprises the following steps:

[0009] Step 1: According to the monitoring requirements and database type, configure the data table and attribute information to be monitored, including the name of the data table, whether it is a compressed table, decompression algorithm, storage type used, data block size, primary key, etc. The primary key is the target to be detected, and the primary key can be one or more fields.

[0010] Step 2: Find the corresponding table file and the location of the data in the table file according to the name of the data table configured in step 1, and determine the file data of the database.

[0011] If the data content of the data block size is a variable size block, you need to first read the first 8 bytes of the data block header, and then extract the data content specified by the DataLen field in the VarBlock data of the variable block; if the content in the data block is compressed, use the corresponding decompression algorithm configured in step 1 to decompress the data.

[0012] Step 3: Iteratively read the data content of the corresponding data block size in the table file of the database according to the data position found in step 2 and the data block size configured in step 1. The data position is the position of the data in the table file.

[0013] Step 4: Extract the corresponding primary key from the data table that needs to be monitored in step 1 to form a detection table.

[0014] If a column-based storage database is used, each primary key field corresponds to one or more files. The tuple value and tuple sequence number in each segment file are read separately, and then the period attribute value and the primary key attribute value are associated according to the tuple sequence number to form a detection table.

[0015] Step 5: Determine whether there are duplicate primary keys in the test table; count whether there are duplicate primary keys in the test table in step 4.

[0016] If there is a duplicate primary key, the data has been tampered with, and continue with step 6 to output the data change record; if there is no duplicate primary key, end the monitoring.

[0017] Step 6: Determine and extract data change records before and after data tampering, including data name, file name, and primary key value.

[0018] The beneficial effect brought by the present invention is strong universality, and can be applied to the more popular relational databases or distributed databases in the industry, such as Postgresql, Greenplum, MySQL, Mariadb, etc. The method can realize data tampering detection by only reading the database table file, without additional network data transmission, and its security is also relatively high. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The present invention will be further described below in conjunction with the accompanying drawings.

[0020] Figure 1 It is a flowchart of an embodiment of the present invention. DETAILED DESCRIPTION

[0021] The following, with reference to the accompanying drawings, describes the embodiments and further explains the specific implementation methods of the present invention, such as the relative positions and connection relationships between the various parts involved, the functions and working principles of the various parts, and the operation and use methods, so as to help those skilled in the art to have a more complete, accurate and in-depth understanding of the concept and technical solution of the present invention. Example

[0022] In this embodiment, the Greenplum database is used, the version number is V5.20, and the application scenario is a mobile base station performance indicator database, the purpose of which is to be able to identify the modified data of the performance indicator.

[0023] Create a non-compressed column storage data table named AOCS_UMCOP_PM, which contains 4 field values, among which "network element ME" is the partition key, "base station identifier ENBID" and "reporting period RP" are the joint primary keys and RTIME is the period, and "dropped call number CALLDOWN" is the service field. The specific information is shown in Table 1.

[0024]

[0025] Table 1;

[0026] After the data table is created, a record is inserted into the database with a reporting period RP of 8:00 am on December 20, 2021, network element ME number NJJY001, base station identification 58001, and number of dropped calls 100. The corresponding query data table record is shown in Table 2.

[0027]

[0028] Table 2;

[0029] Then change the "number of dropped calls" in the above data record to 50, and the corresponding data table query record is shown in Table 3.

[0030]

[0031] Table 3;

[0032] The last query did not show that the “Number of Dropped Calls” was changed to 50.

[0033] like Figure 1 The method for monitoring data tampering based on append mode checking is shown. If the data tampering needs to be found, the present invention includes the following steps:

[0034] Step 1: The data table in this embodiment is AOCS_UMCOP_PM, the attribute information of the data table is a non-compressed table, and the storage type adopted by the data table is column storage.

[0035] The specific information of the data table used in this embodiment is shown in Table 1.

[0036]

[0037] Table 1;

[0038] Step 2: Since the data table uses column storage, as a conventional technical means, each field in the database using column storage corresponds to one or more segment files.

[0039] In the Greenplum database directory, the storage segment files corresponding to each column in the data table AOCS_UMCOP_PM are as follows: Table 4:

[0040]

[0041] Table 4;

[0042] Step three: according to the position of the data found in step two in the table file, that is, the data position and the data block size configured in step one, iteratively read the data content of the corresponding data block size in the table file of the database, as described in detail in this embodiment as follows.

[0043] For the column-based storage segment file of the Greenplum database, you need to read the first 8 bytes of its data block header and then extract the data of the size DataLen of the variable block VarBlock data. For example, after the first piece of data is inserted, the binary data of the segment file corresponding to the "Network Element ME" field is as follows.

[0044] 00000000 00 04 00 19 00 00 00 03 ff e9 d6 8b ca 3b 41 a2 |.............;A.|

[0045] 00000010 01 00 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 |................|

[0046] 00000020 00 00 00 00 08 00 00 00 88 4e 4a 4a 59 30 30 31 |.........NJJY001|

[0047] Step 4: In this embodiment, a column storage database is used, and each primary key field corresponds to one or more files. The tuple value and tuple sequence number in each segment file are read respectively, and then the period attribute value and the primary key attribute value are associated according to the tuple sequence number to form a detection table.

[0048] In this embodiment, the binary data of the segment file corresponding to the "base station identifier ENBID" field is as follows:

[0049] 00000000 00 04 00 19 00 00 c0 02 b8 33 ba 88 38 cd 76 31 |.........3..8.v1|

[0050] 00000010 01 00 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 |................|

[0051] 00000020 00 00 00 00 06 00 00 00 86 35 38 30 30 31 00 00 |.........58001..|

[0052] 00000030 00 04 00 19 00 00 c0 02 bc ec 2c 8c 22 e1 46 50 |..........,.".FP|

[0053] 00000040 65 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 |e.............|

[0054] 00000050 00 00 00 00 06 00 00 00 86 35 38 30 30 31 00 00 |.........58001..|

[0055] The bytes "35 38 30 30 31" are the hexadecimal representation of two repeated "Base Station Identification ENBID" bytes.

[0056] In this embodiment, the binary data of the segment file corresponding to the "Reporting Period RP" field is as follows:

[0057] 00000000 00 04 00 19 00 00 00 03 c0 b5 4d e6 34 9d 17 d1 |..........M.4...|

[0058] 00000010 01 00 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 |................|

[0059] 00000020 00 00 00 00 08 00 00 00 00 20 46 0b 8e 76 02 00 |......... F..v..|

[0060] 00000030 00 04 00 19 00 00 00 03 c4 6a db e2 2e b1 27 b0 |.........j....'.|

[0061] 00000040 65 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 |e.............|

[0062] 00000050 00 00 00 00 08 00 00 00 00 20 46 0b 8e 76 02 00 |......... F..v..|

[0063] The byte "00 20 46 0b 8e 76 02 00" is the hexadecimal data of the "Reporting Period RP" field, which occupies 8 bytes and uses the little-endian representation, with the low-order byte first and the high-order byte last. "00 20 46 0b 8e 76 02 00" represents the number of seconds from 8:00 a.m. on December 20, 2021 to the year 2000.

[0064] In this embodiment, the binary data of the segment file corresponding to the "number of call drops CALLDOWN" field is as follows:

[0065] 00000000 00 04 00 19 00 00 80 02 25 92 51 8c 94 2b 81 0e |........%.Q..+..|

[0066] 00000010 01 00 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 |................|

[0067] 00000020 00 00 00 00 04 00 00 00 64 00 00 00 00 00 00 00 |.....d....|

[0068] 00000030 00 04 00 19 00 00 80 02 84 f7 d4 a5 71 47 1b b5 |............qG..|

[0069] 00000040 65 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 |e.............|

[0070] 00000050 00 00 00 00 04 00 00 00 32 00 00 00 00 00 00 00 |........2.......|

[0071] "64 00 00 00" and "32 00 00 00" are the hexadecimal data of the "CALLDOWN" field, using little-endian representation, with the low-order byte first and the high-order byte last. "64 00 00 00" is 100 in decimal, and "32 00 00 00" is 50 in decimal.

[0072] Step 5: Determine whether there are duplicate primary keys in the test table; count whether there are duplicate primary keys in the test table in step 4.

[0073] In this embodiment, if there is a duplicate primary key in the "Base Station Identification ENBID" field in step four at 8:00 am on December 20, 2021, and its value is 58001, it means that the data has been tampered with, and continue with step six to output the data change record.

[0074] Step 6: Count the tampered files in this embodiment, which are 57360.1, 57360.129, 57360.257 and 57360.385. The extracted data before and after the tampering is recorded as the "CALLDOWN" field. The value before modification is 100 and after modification is 50.

[0075] The present invention is not limited to the above embodiments, and any technical solutions formed by equivalent replacement shall fall within the protection scope required by the present invention.

Claims

1. A method for monitoring data tampering based on append mode inspection, characterized in that The following steps are involved: Step 1: Configure the data tables and attribute information to be monitored according to the monitoring requirements and database type; The data table and attribute information include the name of the data table, whether it is a compressed table, the decompression algorithm, the storage type used, the data block size and the primary key; Step 2: Find the corresponding table file and the location of the data in the table file according to the name of the data table configured in step 1, and determine the file data of the database; Step 3: According to the data position found in step 2 and the data block size configured in step 1, iteratively read the data content of the corresponding data block size in the table file of the database; Step 4: extract the corresponding primary key in the data table to be monitored in step 1 to form a detection table; Step 5: Determine whether there are duplicate primary keys in the detection table; count whether there are duplicate primary keys in the detection table in step 4; if there are duplicate primary keys, the data has been tampered with, and continue to step 6 to output data change records; if there are no duplicate primary keys, end the monitoring; Step 6: Determine and extract the data change records before and after data tampering, including data name, file name and primary key value.

2. The method for monitoring data tampering based on append mode check according to claim 1 is characterized in that: The primary key is a target to be detected, and the primary key is one or more fields.

3. The method for monitoring data tampering based on append mode check according to claim 1, characterized in that: The decompression algorithms include the zlib algorithm and the quicklz algorithm.

4. The method for monitoring data tampering based on append mode check according to claim 1 is characterized in that: In the file data of the database, if the database adopts row storage, each data table corresponds to one or more heap files; if the database adopts column storage, each field corresponds to one or more segment files.

5. The method for monitoring data tampering based on append mode check according to claim 1, characterized in that: In the step three, the data position is the position of the data in the table file.

6. The method for monitoring data tampering based on append mode checking according to claim 1 is characterized in that: In the step 4, if a row-based storage database is used, the tuple values ​​in each file are read respectively to form a detection table.