Application-awareness-free data encryption storage system and method
By configuring the interception and transformation of SQL statements at the application layer, data is encrypted and decrypted dynamically, and the problems of low development efficiency, high maintenance cost and performance impact in the existing technology are solved, and efficient data encryption without perception is achieved.
Patent Information
- Application Number
- CN202411092413.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-09
- Publication Date
- 2024-12-13
AI Technical Summary
The prior art has low development efficiency and high maintenance costs in data encryption storage, and the encryption and decryption code intrudes into business logic that affects maintainability, and there is a risk of integration omissions, and the encryption and decryption of the entire database affects performance.
Implement the encryption engine at the application layer, and dynamically encrypt and decrypt sensitive data in SQL statements through configuration intercepting and transforming SQL statements. Multi-stage configuration and rewrite modules are adopted to reduce developers' concerns and improve the efficiency of encrypting and decrypting integration.
It realizes unconscious data encryption, reduces development and maintenance costs, avoids encrypted and decrypted codes from intruding into business logic, and improves integration efficiency and system performance.
Smart Images
Figure CN119128927A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data encryption technology, and in particular to an application-unaware data encryption storage system and method. Background Art
[0002] In recent years, data security issues have occurred frequently. When the database is attacked and leaked, the company's key internal data will be exposed. Enterprise data security is very important, so it is necessary to implement encryption when storing data in the storage layer.
[0003] In order to solve these problems, the current main method is to implement encryption and decryption by manually hard-coding the data to be encrypted and stored in each business module code at the application layer. In this way, the encryption and decryption code will be invaded into the business logic code. The problems with this method are: one is low development efficiency, the second is high subsequent maintenance costs, and the third is that the encryption and decryption code invades the business logic code, which will affect the maintainability of the business logic code and improve the development efficiency of developers. Usually, the operation logic code of a table in the business logic code will be scattered in different code files, which makes the encryption and decryption logic have the risk of integration omissions.
[0004] Another method is to use encryption devices to encrypt and decrypt all operations on the entire database. This method is extremely costly and will affect the overall database performance. Summary of the invention
[0005] The purpose of the present invention is to provide an application-unaware data encryption storage system and method. The present invention mainly utilizes an encryption engine implemented at the application layer, intercepts the executed SQL statements through configuration, and then dynamically transforms the SQL statements to encrypt and decrypt sensitive data in the SQL statements, which can reduce the focus of developers and improve the integration efficiency of encryption and decryption.
[0006] In order to achieve the above purpose, the following technical solutions are adopted:
[0007] An application-unaware data encryption storage system includes a client plug-in, wherein the client plug-in is used to implement an encryption and decryption engine, wherein the encryption and decryption engine includes
[0008] A parsing module, which is used to parse tables and fields in SQL statements and determine the type of SQL statements;
[0009] A configuration module, which is used to configure tables and fields that need to be encrypted and decrypted;
[0010] The rewriting module is used to rewrite the SQL statement according to the configuration of the configuration module to encrypt and decrypt the data.
[0011] Furthermore, the parsing module decomposes the SQL statement into the type, table and field of the SQL statement through a lexical parser and a syntax parser, and puts the parsed result into the parsing context, wherein the types of SQL statements include UPDATE, INSERT, DELETE and SELECT.
[0012] Furthermore, the rewriting module rewrites the SQL statement based on the parsing context parsed by the parsing module.
[0013] Furthermore, the configuration module adopts a multi-stage configuration method, including the following four stages:
[0014] Double writing stage: the rewriting module rewrites SQL statements of the UPDATE, INSERT, and DELETE types to write both plaintext fields and ciphertext fields;
[0015] Read ciphertext stage: the rewrite module rewrites the SELECT type SQL statement into the ciphertext field;
[0016] Single-write ciphertext stage: If no problems occur in the above two stages after a period of verification, the configuration will be switched to the single-write ciphertext stage. The rewrite module will rewrite SQL statements of the UPDATE, INSERT, and DELETE types into write-only ciphertext fields.
[0017] Plaintext field deletion stage: If there is no problem in performing verification in the single-write ciphertext stage, the rewrite module deletes the plaintext fields in the MYSQL table and leaves the ciphertext fields.
[0018] Furthermore, the configuration module includes
[0019] The encryption and decryption engine master switch is used to control the risks when the encryption and decryption engine is integrated into the project;
[0020] Table-level encryption and decryption configuration, wherein the table-level encryption and decryption configuration is a configuration of a table level that needs encryption and decryption;
[0021] The query uses a ciphertext field configuration, and the query uses a ciphertext field configuration is used to control whether the query operation of the table is switched to use the ciphertext field;
[0022] Double write encryption and decryption configuration, which is used to control the execution rules of the encryption and decryption rewrite SQL engine;
[0023] Field-level encryption and decryption configuration, which is used to set encryption and decryption for specific fields in a table;
[0024] A plaintext field and a ciphertext field mapping configuration is configured, and the plaintext field and the ciphertext field mapping configuration is used to configure the ciphertext field corresponding to the plaintext field.
[0025] An application-unaware data encryption storage method includes the above system and specifically comprises the following steps:
[0026] Intercept the executing SQL statement;
[0027] Parse the tables, fields, and types of SQL statements in SQL statements through the parsing module;
[0028] According to the configuration of the configuration module and the analysis results of the analysis module, the SQL statement is rewritten through the rewriting module to perform data encryption and decryption processing.
[0029] By adopting the above scheme, the beneficial effects of the present invention are:
[0030] The present invention mainly utilizes the encryption engine implemented at the application layer, intercepts the executed SQL statements through configuration, and then dynamically transforms the SQL statements to encrypt and decrypt sensitive data in the SQL statements, which can reduce the focus of developers and improve the integration efficiency of encryption and decryption. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] Figure 1 It is a principle block diagram of the present invention;
[0032] Figure 2 It is a flow chart of the present invention. DETAILED DESCRIPTION
[0033] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments.
[0034] Reference Figures 1 to 2 As shown, the present invention provides an application-unaware data encryption storage system, including a client plug-in, the client plug-in is used to implement an encryption and decryption engine. In one embodiment, the encryption and decryption engine includes
[0035] A parsing module, which is used to parse tables and fields in SQL statements and determine the type of SQL statements;
[0036] A configuration module, which is used to configure tables and fields that need to be encrypted and decrypted;
[0037] The rewriting module is used to rewrite the SQL statement according to the configuration of the configuration module to encrypt and decrypt the data.
[0038] In the present invention, firstly, an encryption and decryption engine is implemented based on a client plug-in, and the client plug-in is implemented as a proxy for the MYSQL connection driver. The main functions of the encryption and decryption engine are: first, parsing SQL statements, and second, rewriting SQL statements through encryption and decryption algorithms, and then implementing a configuration module to configure tables and fields that need to be encrypted and decrypted. When the service is started, the encryption and decryption engine will load the configuration and complete the initialization of the encryption and decryption engine. Any SQL operation will pass through the encryption and decryption engine, that is, first, parsing the tables and fields in the SQL statements, matching the tables and fields that need to be encrypted and decrypted, and then rewriting the SQL statements according to the encryption and decryption algorithms used by the configuration module.
[0039] The parsing module decomposes the SQL statement into the type, table and field of the SQL statement through a lexical parser and a syntax parser, and puts the parsed result into the parsing context, wherein the types of SQL statements include UPDATE, INSERT, DELETE and SELECT.
[0040] In the application service, when any SQL statement is executed, the parsing module first decomposes the SQL statement into the type, table, and field through the lexical and grammatical parsers. After the type is decomposed, it will be concluded that the SQL statement is one of the types of UPDATE, INSERT, DELETE, and SELECT. Then, the tables and fields involved in the SQL statement are parsed. There may be multiple tables here, such as an SQL statement with a SELECT...JOIN... structure. Finally, all parsing results are put into the parsing context for use by the next module, that is, the rewrite module.
[0041] In one embodiment, the rewriting module rewrites the SQL statement based on the parsing context obtained by parsing the parsing module.
[0042] The parsing module rewrites the SQL according to the obtained parsing context. Specifically:
[0043] If it is an UPDATE, INSERT, or DELETE type SQL statement, the plaintext field needs to be encrypted. For UPDATE and INSERT type SQL statements, a ciphertext field will be added to the SQL statement during the compatibility phase, and then both the plaintext field and the ciphertext field will be written at the same time through double writing, as shown in the following simple example:
[0044] INSERT `test`(id,name,phone)values(1,'test','12302...') will be changed to INSERT `test`(id,name,phone,phone_cipher)values(1,'test','12302...','Jsmxjdl2iaknudso').
[0045] Through the above method, you can perform encrypted double writing without affecting the program, which is zero risk for the production environment.
[0046] If it is a SELECT type SQL statement, the query plaintext field will be rewritten as a ciphertext field in the SQL statement during the ciphertext reading phase, such as:
[0047] SELECT id,name,phone from `test` will be rewritten as SELECT id,name,phone_cipherfrom `test`.
[0048] Through the above method, the SQL that originally reads plain text is rewritten to read cipher text, and finally the cipher text information 'Jsmxjdl2iaknudso' is obtained. Before returning the result to the business logic layer, 'Jsmxjdl2iaknudso' will be decrypted to '12302...', and finally the plain text information will be returned to the business logic layer. In this way, the cipher text can be decrypted without the program being aware of it. At this time, for the production environment, if a problem is found, it is only necessary to adjust the configuration to read the plain text field, then the rewrite logic will not be executed, and it will roll back quickly, greatly reducing the production risk. As for the encryption and decryption algorithm: the present invention uses a general symmetric reversible encryption and decryption algorithm for data encryption and decryption, and the present invention can be used with any encryption and decryption algorithm.
[0049] In one embodiment, the configuration module adopts a multi-stage configuration method, including the following four stages:
[0050] Double writing stage: the rewriting module rewrites SQL statements of the UPDATE, INSERT, and DELETE types to write both plaintext fields and ciphertext fields;
[0051] Read ciphertext stage: the rewrite module rewrites the SELECT type SQL statement into the ciphertext field;
[0052] Single-write ciphertext stage: If no problems occur in the above two stages after a period of verification, the configuration will be switched to the single-write ciphertext stage. The rewrite module will rewrite SQL statements of the UPDATE, INSERT, and DELETE types into write-only ciphertext fields.
[0053] Plaintext field deletion stage: If there is no problem in performing verification in the single-write ciphertext stage, the rewrite module deletes the plaintext fields in the MYSQL table and leaves the ciphertext fields.
[0054] The present invention only requires configuration by the user, and through configuration hot loading, the configuration change can be detected without restarting the running application program, thereby enabling dynamic configuration.
[0055] Through configuration, the program can achieve multi-stage, low-risk and smooth encrypted storage.
[0056] It is mainly divided into four stages: the first is the double-write stage, the second is the ciphertext reading stage, the third is the single-write ciphertext stage, and the fourth is the plaintext field deletion stage.
[0057] Among them, in the double-write stage, the encryption and decryption engine will rewrite SQL statements of the UPDATE, INSERT, and DELETE types to write both plaintext fields and ciphertext fields; in the ciphertext reading stage, the encryption and decryption engine will rewrite SQL statements of the SELECT type to check ciphertext fields; for the single-write ciphertext stage, if the above two stages have been verified for a period of time without any problems, it will be switched to the single-write ciphertext stage through configuration, and the encryption and decryption engine will rewrite SQL statements of the UPDATE, INSERT, and DELETE types to write only ciphertext fields. Finally, for the plaintext field deletion stage, after the third stage of verification is successful, the plaintext fields in the MYSQL table can be deleted, leaving the ciphertext fields.
[0058] In one embodiment, the configuration module includes
[0059] The encryption and decryption engine master switch is used to control the risks when the encryption and decryption engine is integrated into the project;
[0060] Table-level encryption and decryption configuration, wherein the table-level encryption and decryption configuration is a configuration of a table level that needs encryption and decryption;
[0061] The query uses a ciphertext field configuration, and the query uses a ciphertext field configuration is used to control whether the query operation of the table is switched to use the ciphertext field;
[0062] Double write encryption and decryption configuration, which is used to control the execution rules of the encryption and decryption rewrite SQL engine;
[0063] Field-level encryption and decryption configuration, which is used to set encryption and decryption for specific fields in a table;
[0064] A plaintext field and a ciphertext field mapping configuration is configured, and the plaintext field and the ciphertext field mapping configuration is used to configure the ciphertext field corresponding to the plaintext field.
[0065] The encryption and decryption engine master switch (data-encrypt:enable) is the master switch of the encryption and decryption engine, which is used to control the risks when the project integrates the encryption and decryption engine; the table-level encryption and decryption configuration (data-encrypt:tables) is the table-level configuration that needs to be encrypted and decrypted; the query uses the ciphertext field configuration (user_unify:queryWithCipherColumn) to control whether the query of the table is switched to use the ciphertext field. This configuration is used when the table has sensitive plaintext fields of historical data. After the plaintext data cleaning tool completes the cleaning, this configuration allows the query SQL to use the ciphertext field smoothly, so as to achieve the effect of no system downtime during the switching period and no perception by the system users; the double-write encryption and decryption configuration (us er_unify:doubleWrite) is used to control the execution rules of the SQL engine for changing encryption and decryption. Enabling double-write encryption and decryption will rewrite SQL to write both plaintext fields and ciphertext fields. This can avoid writing only ciphertext fields. If a problem occurs and the data cannot be rolled back, the risk of field switching is greatly reduced. Field-level encryption and decryption configuration (user_unify:column) is the configuration of the field level that needs to be encrypted and decrypted. Plaintext field and ciphertext field mapping configuration (user_unify:column:email:cipherColumn) is used to configure the ciphertext field corresponding to the plaintext field email (email is a real database table field name). The encryption and decryption engine rewrites SQL through this configuration.
[0066] In addition, a data encryption storage method without application awareness is also provided, including the above system, and specifically including the following steps:
[0067] Intercept the executing SQL statement;
[0068] Parse the tables, fields, and types of SQL statements in SQL statements through the parsing module;
[0069] According to the configuration of the configuration module and the analysis results of the analysis module, the SQL statement is rewritten through the rewriting module to perform data encryption and decryption processing.
[0070] This method achieves unified encryption and decryption through non-invasive configuration. It identifies whether the current SQL needs encryption and decryption based on the configuration at the data layer. There is no problem of code omissions leading to encryption and decryption integration omissions.
[0071] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.
Claims
1. An application-unaware data encryption storage system, comprising a client plug-in, wherein the client plug-in is used to implement an encryption and decryption engine, characterized in that: The encryption and decryption engine includes A parsing module, which is used to parse tables and fields in SQL statements and determine the type of SQL statements; A configuration module, which is used to configure tables and fields that need to be encrypted and decrypted; The rewriting module is used to rewrite the SQL statement according to the configuration of the configuration module to encrypt and decrypt the data.
2. The application-unaware data encryption storage system according to claim 1, characterized in that: The parsing module decomposes the SQL statement into the type, table and field of the SQL statement through a lexical parser and a syntax parser, and puts the parsed result into the parsing context, wherein the types of SQL statements include UPDATE, INSERT, DELETE and SELECT.
3. The application-unaware data encryption storage system according to claim 2, characterized in that: The rewriting module rewrites the SQL statement based on the parsing context parsed by the parsing module.
4. The application-unaware data encryption storage system according to claim 1, characterized in that: The configuration module adopts a multi-stage configuration method, including the following four stages: Double writing stage: the rewriting module rewrites SQL statements of the UPDATE, INSERT, and DELETE types to write both plaintext fields and ciphertext fields; Read ciphertext stage: the rewrite module rewrites the SELECT type SQL statement into the ciphertext field; Single-write ciphertext stage: If no problems occur in the above two stages after a period of verification, the configuration will be switched to the single-write ciphertext stage. The rewrite module will rewrite SQL statements of the UPDATE, INSERT, and DELETE types into write-only ciphertext fields. Plaintext field deletion stage: If there is no problem in performing verification in the single-write ciphertext stage, the rewrite module deletes the plaintext fields in the MYSQL table and leaves the ciphertext fields.
5. The application-unaware data encryption storage system according to claim 4, characterized in that: The configuration module includes The encryption and decryption engine master switch is used to control the risks when the encryption and decryption engine is integrated into the project; Table-level encryption and decryption configuration, wherein the table-level encryption and decryption configuration is a configuration of a table level that needs encryption and decryption; The query uses a ciphertext field configuration, and the query uses a ciphertext field configuration is used to control whether the query operation of the table is switched to use a ciphertext field; Double write encryption and decryption configuration, which is used to control the execution rules of the encryption and decryption rewrite SQL engine; Field-level encryption and decryption configuration, which is used to set encryption and decryption for specific fields in the table; plaintext field and ciphertext field mapping configuration, which is used to configure the ciphertext field corresponding to the plaintext field.
6. An application-unaware data encryption storage method, comprising the system according to any one of claims 1 to 5, characterized in that: The steps include: Intercept the executing SQL statement; Parse the tables, fields, and types of SQL statements in SQL statements through the parsing module; According to the configuration of the configuration module and the analysis results of the analysis module, the SQL statement is rewritten through the rewriting module to perform data encryption and decryption processing.