Column-level transparent encryption method implemented based on openGauss database column storage

Abstract

The invention relates to a column-level transparent encryption method realized based on openGauss database column storage. The method comprises the following steps of: adding a grammar keyword, and marking a specified column as an encrypted column by utilizing the grammar keyword when a column storage table is created; adding a non-shared system table for storing key information corresponding to the encrypted column; when data are inserted into the column storage table in batches, key information is matched in a non-shared system table according to an encryption column, the key information is set to tde info of a CU header, CU data are encrypted by using the key information, and a CU is constructed by using the encrypted data and written into a disk; and when the encrypted CU is read, decrypting the encrypted CU according to the key information stored in the CU header to obtain plaintext CU data. According to the method, the specified column of the column storage table in the openGauss database can be encrypted, so that the purpose of data protection is achieved, and in addition, the data are still plaintext data in the memory, so that the original data query processing process is not influenced at all.

Description

Technical Field

[0001] The present invention belongs to the technical field of data encryption methods, and specifically relates to a column-level transparent encryption method, system and readable storage medium based on openGauss database column storage. Background Art

[0002] like Figure 1 As shown in the figure, in the data organization structure of the openGauss database, the basic storage unit of the column storage engine is CU (Compress Unit), which is a compressed data block composed of a part of the data in a column in the table. The entire table is divided into several CUs according to different columns and managed in units of CU. Using CU data management has many advantages. For example, CU data is suitable for compression and has a high compression ratio, which can save a lot of disk space; CU data is more convenient for data encoding and query; in multi-table queries, if only a few columns are accessed, the amount of data to be read can be greatly reduced in this scenario, greatly reducing IO consumption and improving query performance; vector operations based on column batch data, combined with the vectorized execution engine, the CPU cache hit rate is higher, and the OLAP performance is better.

[0003] However, CU data management also has certain defects. Since it can bypass the database management mechanism and directly obtain plaintext data from disk files, there is a serious risk of data leakage. In order to ensure data security, it is necessary to develop corresponding data protection measures. Summary of the invention

[0004] In order to overcome the data leakage risk in CU data management, this paper proposes a new column-level transparent encryption method based on openGauss database column storage. The purpose of this invention is to provide a column-level transparent encryption strategy based on openGauss database column storage, through which the specified columns of the column storage table in the openGauss database can be encrypted, thereby achieving the purpose of data protection.

[0005] In summary, the method of the present invention allows users to choose to encrypt sensitive data columns and store them in disk files after encryption, thereby avoiding data leakage problems caused by disk files. In addition, since the data in the memory is still plain text, it will not have any impact on the original data query processing process.

[0006] Specifically, the present invention provides a column-level transparent encryption method based on openGauss database column storage implementation, such as Figure 4 As shown, the method comprises the following steps: S1. Add a syntax keyword and use the syntax keyword to mark the specified column as an encrypted column when creating a column storage table; S2. Add a non-shared system table to store the key information corresponding to the encrypted column; S3. When inserting data in batches into a column-stored table, match the key information of the encrypted column to the non-shared system table, set the key information to the tde info of the CU header, use the key information to encrypt the CU data, construct the CU using the encrypted data, and write it to the disk; S4. When reading the encrypted CU, the encrypted CU is decrypted according to the key information stored in the CU header to obtain the plaintext CU data.

[0007] Furthermore, the fields stored in the non-shared system table in step S2 of the column-level transparent encryption method implemented based on openGauss database column storage of the present invention include but are not limited to: user table oid, serial number of the encrypted column in the user table, key ciphertext, key check value, master key id, and algorithm type.

[0008] Furthermore, in the column-level transparent encryption method implemented based on the openGauss database column storage of the present invention, the encryption mark of the encrypted column is recorded in the attextraopt of the corresponding table id column record in the non-shared system table.

[0009] Furthermore, in the column-level transparent encryption method implemented based on the openGauss database column storage of the present invention, key information is distributed when the encrypted column data is written in batches to the column storage table for the first time.

[0010] Furthermore, the column-level transparent encryption method implemented based on openGauss database column storage of the present invention includes step S3: S31. When inserting data in batches into a column storage table, first determine whether the current column is an encrypted column; S32. If it is an encrypted column, query the key information corresponding to the column in the non-shared system table; S33. If the corresponding key information is found, it is used directly; S34. If the corresponding key information is not found, the key information is generated and stored in a non-shared system table; S35. Generate an encryption initialization vector; S36 sets the key information to tde info of CU header; S37 uses the key information to encrypt the CU data to obtain the encrypted data; S38. Construct a CU using the encrypted data and write it to disk.

[0011] On the other hand, the present invention also provides a column-level transparent encryption system based on openGauss database column storage. When the transparent encryption system is running, the steps of the column-level transparent encryption method based on openGauss database column storage are implemented.

[0012] In addition, the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the steps of the column-level transparent encryption method based on openGauss database column storage are implemented.

[0013] In summary, the column-level transparent encryption method implemented based on the openGauss database column storage of the present invention allows users to choose to encrypt sensitive data columns and store them in disk files after encryption, thereby avoiding the data leakage problem caused by disk files. This method can encrypt the specified columns of the column storage table in the openGauss database, thereby achieving the purpose of data protection. Moreover, since the data in the memory is still plaintext, it will not have any impact on the original data query processing process. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the background technology and the technical solution of the present invention, the following briefly introduces the drawings required for use in the description of the present invention. Obviously, the following drawings are only some embodiments recorded in the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without paying any creative work.

[0015] Figure 1 This is a schematic diagram of the storage structure of the openGauss database column storage file.

[0016] Figure 2 This is a schematic diagram of encrypted CU storage in the method of the present invention.

[0017] Figure 3 The encrypted operation flow diagram of the method of the present invention is shown in FIG.

[0018] Figure 4 The figure is a flow chart of the overall implementation of the method of the present invention. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. The present invention can also be implemented or applied through other different specific implementation methods. The details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention.

[0020] At the same time, it should be understood that the protection scope of the present invention is not limited to the specific embodiments described below; it should also be understood that the terms used in the embodiments of the present invention are for describing specific embodiments rather than for limiting the protection scope of the present invention.

[0021] Embodiment: A column-level transparent encryption method based on openGauss database column storage In this embodiment, the keyword ENCRYPT is added to the syntax to identify the specified column as an encrypted column when creating a column storage user table. A non-shared system table vb_security_key_info is added to store the key information corresponding to the encrypted column. The fields include: reloid (user table oid), attnum (serial number of the encrypted column in the user table), keyinfo (key cipher (key ciphertext and key check value), masterkeyid (master key id), algorithm (algorithm type).

[0022] First, create a column-stored table and mark the specified columns as encrypted columns: CREATE TABLE table1(id int ENCRYPT, info varchar(2)) with(ENABLE_TDE=ON,ENCRYPT_ALGO=SM4_CTR,ORIENTATION=COLUMN); In this syntax, ENABLE_TDE=ON indicates that transparent encryption is enabled for the table, ORIENTATION=COLUMN indicates that the table is a column-based table, ENCRYPT_ALGO=SM4_CTR indicates that all columns of the table use the SM4_CTR algorithm, the id column is marked as ENCRYPT, indicating that the column is an encrypted column, and the info column without the ENCRYPT mark indicates that the column is a non-encrypted column.

[0023] The options in with are saved in the options field of the corresponding table1 record in the pg_class system table. The encryption mark for the id column is recorded in the attextraopt field of the corresponding table1 id column record in the pg_attribute system table.

[0024] At this time, no key has been assigned to the id column, so the key information of the id column does not exist in the system table vb_security_key_info. The key is assigned when the id column data is written to the table in batches for the first time. For detailed process, see the following batch data insertion process.

[0025] like Figure 3As shown in the figure, when inserting data into a table in batches, the following process is used for processing: 1) Determine whether the current column is an encrypted column; 2) If it is an encrypted column, query the key information corresponding to the column in the system table; 3) Use the key information directly after querying it; 4) If no key information is found, generate key information and store it in the system table; 5) Generate encryption initialization vector IV; 6) Set the key information to the tde info (tde: TransparentDataEncryption) of the CU header, such as Figure 2 As shown; 7) Encrypt the CU data using the key information to obtain the encrypted data; 8) Use the encrypted data to construct a CU and write it to disk; 9) When reading the encrypted CU, it is decrypted according to the key information stored in the CU header to obtain the plaintext CU data.

[0026] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Any technician familiar with the profession can make some changes or modifications to the technical contents disclosed above without departing from the scope of the technical solution of the present invention to obtain equivalent embodiments of equivalent changes. However, any simple modification, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention should be included in the scope of protection of the claims of the present invention.

Claims

1. A column-level transparent encryption method based on openGauss database column storage, characterized in that: The method comprises: S1. Add a syntax keyword and use the syntax keyword to mark the specified column as an encrypted column when creating a column storage table; S2. Add a non-shared system table to store the key information corresponding to the encrypted column; S3. When inserting data in batches into a column-stored table, match the key information of the encrypted column to the non-shared system table, set the key information to the tde info of the CU header, use the key information to encrypt the CU data, construct the CU using the encrypted data, and write it to the disk; S4. When reading the encrypted CU, the encrypted CU is decrypted according to the key information stored in the CU header to obtain the plaintext CU data.

2. The column-level transparent encryption method based on openGauss database column storage according to claim 1 is characterized in that: The fields stored in the non-shared system table in step S2 include: user table oid, serial number of the encrypted column in the user table, key ciphertext, key check value, master key id, and algorithm type.

3. The column-level transparent encryption method based on openGauss database column storage according to claim 2 is characterized in that: The encryption mark of the encrypted column is recorded in the attextraopt of the corresponding table id column record in the non-shared system table.

4. The column-level transparent encryption method based on openGauss database column storage according to claim 2 is characterized in that: Key information is allocated when encrypted column data is written in batches to a column store table for the first time.

5. The column-level transparent encryption method based on openGauss database column storage according to claim 1 is characterized in that: Step S3 includes: S31. When inserting data in batches into a column storage table, first determine whether the current column is an encrypted column; S32. If it is an encrypted column, query the key information corresponding to the column in the non-shared system table; S33. If the corresponding key information is found, it is used directly; S34. If the corresponding key information is not found, the key information is generated and stored in a non-shared system table; S35. Generate an encryption initialization vector; S36 sets the key information to tde info of CU header; S37 uses the key information to encrypt the CU data to obtain the encrypted data; S38. Construct a CU using the encrypted data and write it to disk.

6. A column-level transparent encryption system based on openGauss database column storage, characterized in that: The transparent encryption system implements the steps of the column-level transparent encryption method based on openGauss database column storage as described in any one of claims 1 to 5 when running.

7. A computer-readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the steps of the column-level transparent encryption method based on openGauss database column storage as described in any one of claims 1 to 5 are implemented.

Images