Data encryption query method and device, equipment and storage medium

Abstract

The invention relates to a data encryption query method and device, equipment and a storage medium. The method comprises the following steps: analyzing an SQL request, and obtaining a column field and an information table corresponding to encrypted data carried in the SQL request based on the analyzed SQL request; performing SQL rewriting on the column field and the information table to obtain a rewritten SQL request; performing fragmentation operation on the rewritten SQL request to obtain a target fragment, and sending the fragmented SQL request to a plurality of storage nodes corresponding to the target fragment; obtaining service information stored in the corresponding storage node according to the rewritten SQL request, determining encrypted data in the service data based on the metadata table and the MySQL protocol packet, and decrypting the encrypted data to obtain decrypted data corresponding to the encrypted data; and packaging the decrypted data and the unencrypted data into response data and returning the response data. Compared with the prior art, the data encryption and decryption process is not perceived by a user, and the storage node corresponding to the target fragment can be quickly positioned, so that quick query of mass data is realized.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a data encryption query method, device, equipment and storage medium. Background Art

[0002] With the rapid development of information technology, the growth rate of business data has increased exponentially. The traditional single-database single-table architecture faces performance bottlenecks, poor scalability, and maintenance difficulties when facing massive amounts of data. In order to meet these challenges, the industry has gradually reached a consensus to achieve database-table splitting through database middleware, which is the so-called sharding or database-table splitting technology. This technology splits the data originally stored in a database into multiple parts according to certain rules and stores them in different databases or tables, thereby achieving horizontal expansion of data and improving the system's concurrent processing capabilities and query efficiency.

[0003] Database middleware plays a key role in database-table splitting. It is located between the application layer and the database layer and is responsible for data routing, aggregation, sharding, and failover. Through the middleware, the application layer can operate multiple split databases, which not only simplifies the development of the application layer, but also improves the scalability and flexibility of the system.

[0004] However, after the database and table are split, the data is scattered across multiple databases or tables. The existing tablespace encryption method for the entire database or table cannot perform fine-grained encryption on the split data, resulting in low security of sensitive data during transmission and storage. Summary of the invention

[0005] Based on this, it is necessary to provide a data encryption query method, device, equipment and storage medium that can encrypt the split data in order to solve the above technical problems.

[0006] In a first aspect, the present application provides a data encryption query method, the method comprising:

[0007] Receiving an SQL request, parsing the SQL request, matching the parsed SQL request with a configured metadata table, and obtaining a column field and an information table corresponding to the encrypted data carried in the SQL request;

[0008] Rewrite the column fields and information table in SQL based on the metadata table, embed the encryption algorithm and encryption algorithm parameters in the metadata table into the SQL request, and obtain a rewritten SQL request;

[0009] The rewritten SQL request is fragmented to obtain a target fragment, and the fragmented SQL request is sent to a plurality of storage nodes corresponding to the target fragment; the plurality of storage nodes are distributedly stored with business information and corresponding business data, wherein the business data includes encrypted data and unencrypted data;

[0010] Acquire the business information stored in the corresponding storage node according to the rewritten SQL request, determine the encrypted data in the business data based on the metadata table and the MySQL protocol packet, decrypt the encrypted data, and obtain decrypted data corresponding to the encrypted data;

[0011] The decrypted data and the unencrypted data are encapsulated into response data and returned.

[0012] In one embodiment, before receiving the SQL request, parsing the SQL request, matching the parsed SQL request with the configured metadata table, and obtaining the column fields and information table corresponding to the encrypted data carried in the SQL request, the method further includes:

[0013] A metadata table is configured and stored in a middleware; the metadata table includes encryption fields configured based on encryption data and their corresponding encryption algorithms and encryption algorithm parameters.

[0014] In one embodiment, receiving the SQL request, parsing the SQL request, matching the parsed SQL request with a configured metadata table, and obtaining the column fields and information table corresponding to the encrypted data carried in the SQL request include:

[0015] Receive an SQL request, obtain an SQL statement in the SQL request, and parse the SQL statement to obtain an abstract syntax tree corresponding to the SQL statement;

[0016] Traversing each node of the abstract syntax tree, determining a column name node in each node, and matching the column name node with the encrypted field to obtain a column field corresponding to the encrypted data;

[0017] Based on the matched column field, determine the information table to which the SQL request corresponding to the column field belongs, push the information table into the stack using the stack structure, obtain the information table name of the SQL statement corresponding to the column field at the top of the stack structure, and determine the column field and the information table to which it belongs.

[0018] In one of the embodiments, the types of the SQL request include query request, insert request, update request and / or delete request;

[0019] The step of rewriting the column fields and the information table in SQL based on the metadata table, embedding the encryption algorithm and encryption algorithm parameters in the metadata table into the SQL request, and obtaining the rewritten SQL request includes:

[0020] Acquire the metadata table, and determine the encryption algorithm and encryption algorithm parameters of the column field according to the encrypted field in the metadata table that matches the column field;

[0021] Based on the encryption algorithm, the encryption algorithm parameters and the type of the SQL request, the query request, the insert request, the update request and / or the delete request are rewritten for the column field and the information table to obtain a rewritten SQL request; the rewritten SQL request is embedded with the encryption algorithm and the encryption algorithm parameters.

[0022] In one of the embodiments, the middleware also stores shard configuration information;

[0023] The step of performing a sharding operation on the rewritten SQL request to obtain a target shard, and sending the sharded SQL request to a plurality of storage nodes corresponding to the target shard, includes:

[0024] Obtain sharding configuration information and initialize sharding; the sharding configuration information includes sharding key and sharding algorithm;

[0025] Based on the parsed SQL request, extract the shard key in the SQL request;

[0026] Performing sharding calculation on the sharding key based on the sharding algorithm to obtain sharding data, and determining a target shard matching the sharding data according to the sharding data;

[0027] The rewritten SQL request is sent to multiple storage nodes corresponding to the target shard.

[0028] In one embodiment, sending the rewritten SQL request to multiple storage nodes corresponding to the target shard includes:

[0029] Initialize all storage nodes;

[0030] Selecting a master node for the storage node, and determining other storage nodes except the master node as slave nodes;

[0031] When the SQL request is sent to the target shard, control the master node to perform data encapsulation to encapsulate the SQL request into a log entry and copy it to the master node, and control each slave node to copy the log entry to the corresponding slave node based on the copy request of the master node;

[0032] When the number of slave nodes that replicate the log entry meets a preset condition, the master node is controlled to return the replication result to send the rewritten SQL request to each storage node.

[0033] In one embodiment, the acquiring the business information stored in the corresponding storage node according to the rewritten SQL request, determining the encrypted data in the business data based on the metadata table and the MySQL protocol packet, decrypting the encrypted data, and obtaining the decrypted data corresponding to the encrypted data includes:

[0034] Based on the target shard and the log entries in each storage node, query the business data stored in the storage node, obtain and parse the column part of the MySQL protocol packet, and obtain the column information of each column in the business data;

[0035] Matching the fields of the column information with the encrypted fields in the metadata table, and determining the fields of the column information that successfully match as the decrypted fields;

[0036] Determine the encryption algorithm and encryption algorithm parameters of the decryption field according to the encryption field that successfully matches the decryption field, and determine the corresponding decryption algorithm based on the encryption algorithm and encryption algorithm parameters;

[0037] The decryption field is decrypted based on the decryption algorithm to obtain decrypted data.

[0038] In a second aspect, the present application further provides a data encryption query device, the device comprising:

[0039] A request parsing module, used to receive an SQL request, parse the SQL request, match the parsed SQL request with a configured metadata table, and obtain column fields and information tables corresponding to the encrypted data carried in the SQL request;

[0040] A request rewriting module, used to perform SQL rewriting on the column fields and the information table based on the metadata table, embed the encryption algorithm and encryption algorithm parameters in the metadata table into the SQL request, and obtain a rewritten SQL request;

[0041] A shard storage module is used to perform a sharding operation on the rewritten SQL request to obtain a target shard, and send the sharded SQL request to a plurality of storage nodes corresponding to the target shard; the plurality of storage nodes store business information and its corresponding business data in a distributed manner, and the business data includes encrypted data and unencrypted data;

[0042] A data query module, used to obtain the business information stored in the corresponding storage node according to the rewritten SQL request, and determine the encrypted data in the business data based on the metadata table and the MySQL protocol packet, decrypt the encrypted data, and obtain decrypted data corresponding to the encrypted data;

[0043] The data encapsulation module is used to encapsulate the decrypted data and the unencrypted data into response data and return the response data.

[0044] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above-mentioned method when executing the computer program.

[0045] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the method described above are implemented.

[0046] In a fifth aspect, the present application also provides a computer program product, including a computer program, which implements the steps of the method described above when executed by a processor.

[0047] In the above-mentioned data encryption query method, device, equipment and storage medium, when a user or an application initiates a query, insert, update, delete or other operation on the client, the client converts the operation into a corresponding SQL request, the server receives the SQL request, parses the SQL request, obtains the column fields and information table corresponding to the encrypted data carried in the SQL request based on the parsed SQL request and the configured metadata table, then performs SQL rewriting on the column fields and information table, embeds the encryption algorithm and encryption algorithm parameters in the metadata table into the SQL request, obtains the rewritten SQL request, and performs a sharding operation on the rewritten SQL request to obtain the target shard, and the sharded SQL request The request is sent to multiple storage nodes corresponding to the target shard; finally, the business information stored in the corresponding storage node is obtained according to the rewritten SQL request, the encrypted data in the business data is determined based on the metadata table and the MySQL protocol packet, the encrypted data is decrypted, and the decrypted data corresponding to the encrypted data is obtained, and the decrypted data and the unencrypted data are encapsulated into response data and returned; in this way, while ensuring that the business data queried in the distributed database is consistent and complete, the data transparent encryption and query process is realized, and the user is unaware of the whole process. Moreover, after sharding the rewritten SQL request, the storage node corresponding to the target shard can be quickly located to realize fast query of massive data. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0049] Figure 1 An application environment diagram of a data encryption query method in an embodiment;

[0050] Figure 2 A schematic diagram of a flow chart of a data encryption query method in one embodiment;

[0051] Figure 3 A schematic diagram of a process for obtaining column fields and an information table in one embodiment;

[0052] Figure 4 A schematic diagram of a process of sending an SQL request to a storage node in one embodiment;

[0053] Figure 5 A schematic diagram of a process for performing data decryption in one embodiment;

[0054] Figure 6 It is a structural block diagram of a data encryption query device in one embodiment;

[0055] Figure 7 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0057] The common massive data processing relies only on the mechanism of sharding. Due to the frequent operation of data, it is very easy to encounter distributed asynchrony and delay, which leads to bottleneck problems such as data inconsistency, cross-shard query and distributed performance in distributed scenarios. In addition, when the server of sharding fails, the storage node host partially crashes, or the cluster network fails, serious problems such as loss of business query requests will occur, which seriously affects the query of data in the distributed database system and affects the business. In addition, sharding does not consider the encryption of sensitive data, and realizing transparent data encryption is also an urgent problem to be solved.

[0058] In order to circumvent the above-mentioned data processing and query problems of distributed databases and better meet the performance requirements and security requirements for business data queries in distributed database systems, an embodiment of the present application provides a data encryption query method that can achieve data transparent encryption and query optimization.

[0059] The data encryption query method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. The server 104 can parse the received SQL (Structured Query Language) request, and can obtain the field corresponding to the encrypted data carried in the SQL request and the information table to which the field belongs based on the parsed SQL request, and rewrite the field and information table of the encrypted data, and send the rewritten SQL request to each storage node. The storage node has distributed storage of corresponding business data, and the business data queried by the SQL request is decrypted by the decryption method corresponding to the encryption method of the encrypted data, and the corresponding decrypted data is obtained and returned. The user has no perception of the entire encryption and decryption process. Among them, the terminal 102 can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car devices, projection devices, etc. Portable wearable devices can be smart watches, smart bracelets, etc. The server 104 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services.

[0060] In an exemplary embodiment, Figure 2 As shown, a data encryption query method is provided, which is applied to Figure 1 The server in the example is used to illustrate, including the following steps S201 to S205. Among them:

[0061] Step S201, receiving an SQL request, parsing the SQL request, matching the parsed SQL request with the configured metadata table, and obtaining the column fields and information table corresponding to the encrypted data carried in the SQL request.

[0062] SQL requests refer to instructions sent by users or applications to the server's database to perform specific operations. The characteristic operations corresponding to SQL requests usually include query (SELECT), insert (INSERT), update (UPDATE) and delete (DELETE).

[0063] The metadata table refers to an encryption configuration information table that is pre-configured based on encrypted data and stored in a middleware, and is used to indicate which data is encrypted data and its corresponding encryption algorithms, encryption algorithm parameters, etc.

[0064] Encrypted data refers to sensitive data related to users or applications. Such data can be the user's private data or data that the user considers important and needs to be encrypted.

[0065] Among them, the information table is the basic storage unit in the database, which contains related data rows and columns. Each information table has a specific structure, which defines the type of data that can be stored and the relationship between the data. In an SQL request, usually, an information table refers to a database table from which a user wants to retrieve data. The information table in this embodiment specifically refers to the information table where encrypted data is stored.

[0066] Among them, the column field refers to the column name or field defined in the information table. In the information table, each column represents an attribute or feature of the record in the table. For example, in an information table that stores user information, there may be column fields such as "user name", "telephone number", and "address". In SQL requests, usually, the column field refers to the specific information that the user wants to retrieve from a certain information table. The column field in this embodiment specifically refers to the column name corresponding to the encrypted data. For example, in a certain information table, the user's telephone number is encrypted data, then the column name "telephone number" corresponding to the number is the column field in this embodiment.

[0067] For example, when a user or an application initiates a query, insert, update, delete or other operation on the client, the client converts the operation into a corresponding SQL request, the server responds to and receives the SQL request, and parses the SQL request to obtain a parsed SQL request; subsequently, the server matches the parsed SQL request with a pre-configured metadata table to obtain the column fields and information table corresponding to the encrypted data carried in the SQL request.

[0068] The client may be in the form of a terminal to sense and identify corresponding requests made by users or applications, and convert the requests into SQL requests and send them to the server.

[0069] Step S202, rewrite the column fields and the information table in SQL based on the metadata table, embed the encryption algorithm and encryption algorithm parameters in the metadata table into the SQL request, and obtain a rewritten SQL request.

[0070] Among them, SQL rewriting refers to modifying or converting the existing SQL request statements based on the metadata table, so as to embed the encryption algorithm and encryption algorithm parameters in the metadata table into the rewritten SQL request, so as to encrypt the column fields and their corresponding data in the SQL request.

[0071] The encryption algorithm refers to an algorithm for encrypting data in a column field. The encryption algorithm used in this embodiment includes, but is not limited to, a symmetric encryption algorithm, an asymmetric encryption algorithm, etc. Symmetric encryption algorithms may use algorithms such as AES (Advanced Encryption Standard), DES (Data Encryption Standard), etc., and asymmetric encryption algorithms may use algorithms such as ECC (Elliptic Curve Cryptography), etc.

[0072] The encryption algorithm parameters refer to the parameters required to perform encryption operations on data. Different encryption algorithm parameters may be required. For example, for the AES algorithm, the encryption algorithm parameters may include but are not limited to the key (encryption_key), encryption mode, padding method and other parameters.

[0073] Exemplarily, after obtaining the column fields and information tables associated with the data to be encrypted, the server obtains the encryption algorithm and encryption algorithm parameters associated with the encrypted data in the metadata table, and rewrites the SQL request according to the encryption algorithm and encryption algorithm parameters to embed the encryption algorithm and encryption algorithm parameters into the SQL request to encrypt the data. The user is unaware of the entire encryption process to achieve transparent encryption of the data.

[0074] Step S203, performing a sharding operation on the rewritten SQL request to obtain a target shard, and sending the sharded SQL request to multiple storage nodes corresponding to the target shard.

[0075] The sharding operation refers to the process of splitting the rewritten SQL request into multiple shards in distributed storage or distributed databases. The target shard refers to each small data shard obtained after the sharding operation on the SQL request. Each shard has the same data structure or format, but contains different data content. In distributed storage, each target shard will be stored on a different storage node to achieve distributed storage and management of data (i.e. distributed storage).

[0076] Among them, storage nodes refer to computers or servers responsible for storing and managing data in a distributed storage system. Business information and its corresponding business data are distributedly stored in multiple storage nodes. Business data includes encrypted data and unencrypted data. That is, through target sharding, different storage modules can be respectively targeted, and different business information and business data stored in the storage nodes can be obtained.

[0077] Among them, business information refers to the representation stored in the storage node for indicating the category of business data or for distinguishing different business data, which can be an identifier such as a field, ID or string; business data refers to the data corresponding to the business information, for example, in phone_number='xxx', phone_number is the business information, and 'xxx' is the business data. It can be understood that business information can include encrypted information that needs to be encrypted (i.e., the above-mentioned "column field"), and can also include regular information that does not need to be encrypted; correspondingly, business data can also include encrypted data and unencrypted data.

[0078] Exemplarily, the server performs a sharding operation on the rewritten SQL request to obtain target shards corresponding to various types of data that need to be queried in the SQL request. Each target shard can correspond to a field to query different data. Subsequently, the server sends the sharded SQL request to multiple storage nodes corresponding to the target shards to access the corresponding storage nodes.

[0079] Step S204, obtaining the business information stored in the corresponding storage node according to the rewritten SQL request, and determining the encrypted data in the business data based on the metadata table and the MySQL protocol packet, decrypting the encrypted data, and obtaining decrypted data corresponding to the encrypted data.

[0080] Among them, the MySQL protocol package, also known as a message, refers to the data packet used for communication between the client and the server in the MySQL database (i.e., a relational database management system). It is used to define the data exchange format and rules between the client and the server to ensure the correct execution of communication.

[0081] The decrypted data refers to the data obtained after decrypting the encrypted data stored in the storage node. Usually, the decrypted data and the encrypted data are the same in data content (value).

[0082] Exemplarily, the server obtains the business information stored in the corresponding storage node according to the rewritten SQL request, determines the encrypted data in the business data based on the encrypted fields in the metadata table and the MySQL protocol packet, decrypts the encrypted data, and obtains decrypted data corresponding to the encrypted data.

[0083] Step S205: encapsulate the decrypted data and the unencrypted data into response data and return it.

[0084] The response data refers to data that is queried based on the needs of the user or application; usually, the response data is encapsulated into a MySQL protocol packet and returned.

[0085] Exemplarily, the server integrates the decrypted data and other unencrypted business data into response data, and encapsulates the response data into a MySQL protocol packet and returns it to the client.

[0086] In the above data encryption query method, when a user or an application initiates a query, insert, update, delete or other operation on the client, the client converts the operation into a corresponding SQL request, the server receives the SQL request, parses the SQL request, obtains the column fields and information table corresponding to the encrypted data carried in the SQL request based on the parsed SQL request and the configured metadata table, then rewrites the column fields and information table with SQL, embeds the encryption algorithm and encryption algorithm parameters in the metadata table into the SQL request, obtains the rewritten SQL request, and performs a sharding operation on the rewritten SQL request to obtain the target shard, and sends the sharded SQL request to the server that is connected to the SQL request. Multiple storage nodes corresponding to the target shard; finally, the business information stored in the corresponding storage node is obtained according to the rewritten SQL request, the encrypted data in the business data is determined based on the metadata table and the MySQL protocol packet, the encrypted data is decrypted, and the decrypted data corresponding to the encrypted data is obtained, and the decrypted data and the unencrypted data are encapsulated into response data and returned; in this way, while ensuring the consistency and completeness of the business data queried in the distributed database, the data transparent encryption and query process is realized, and the user is unaware of the whole process. In addition, after sharding the rewritten SQL request, the storage node corresponding to the target shard can be quickly located to realize fast query of massive data.

[0087] In one embodiment, before receiving the SQL request in step S201, parsing the SQL request, matching the parsed SQL request with the configured metadata table, and obtaining the column fields and information table corresponding to the encrypted data carried in the SQL request, the following steps are also included:

[0088] A metadata table is configured and stored in a middleware.

[0089] The metadata table includes encryption fields configured based on encrypted data and their corresponding encryption algorithms and encryption algorithm parameters. The encryption field refers to an identifier used to indicate information such as the type or attribute of the encrypted data, such as phone_number.

[0090] Among them, middleware refers to software that provides connections between system software and application software and facilitates communication between software components, allowing applications to transparently operate the database. The data in the database is stored in different local databases (storage nodes). Logically, these data are a unified whole, but physically they are stored in different storage nodes. In this way, the middleware plays the role of a proxy to store data in a distributed manner.

[0091] Exemplarily, the server configures the encryption fields, encryption algorithms and encryption algorithm parameters of the encrypted data according to the sensitive data determined by the user or the system, that is, the data to be encrypted, forms a metadata table, and stores it in the middleware. In this embodiment, the middleware can be implemented using Zookeeper.

[0092] In an optional embodiment, the metadata table may also include permission allocations of middleware users, that is, which data the middleware can store or access, to further improve the efficiency of data encryption, decryption and query.

[0093] In this embodiment, the metadata table is configured so that the encrypted data can be customized, thereby improving the flexibility of the data encryption process.

[0094] In one embodiment, Figure 3 As shown, obtaining the column fields and information table in step S201 includes the following steps S301 to S303. Among them:

[0095] Step S301, receiving an SQL request, obtaining an SQL statement in the SQL request, and parsing the SQL statement to obtain an abstract syntax tree corresponding to the SQL statement.

[0096] Among them, SQL statements refer to sentences used to express SQL requests. SQL statements are composed of a series of keywords, identifiers, operators and other parameters organized according to SQL syntax rules. For example, SELECT * FROM user_info WHEREphone_number = 'xxx'; among them, SELECT is an operator, meaning "query", that is, a query request, FROM is a keyword used to specify the source of data to be queried, WHERE is a condition used to filter data and retrieve specific information, user_info is the name of the information table, phone_number is a column field, that is, user_info and phone_number are identifiers, and 'xxx' is the value of phone_number, that is, the data to be encrypted.

[0097] Among them, the Abstract Syntax Tree (AST) refers to an intermediate representation generated after the source code undergoes lexical analysis and syntactic analysis. It represents the grammatical structure of the program in the form of a tree.

[0098] For example, when a user or application initiates a SQL request at the application layer, the server obtains a corresponding SQL statement based on the SQL request, and sends the SQL statement to a middleware or query engine for processing to parse the SQL statement and obtain a corresponding abstract syntax tree, which represents each component of the SQL statement in a tree structure.

[0099] In an optional example, take querying the phone number phone_number in the user information table user_info as an example; the server converts the SQL request obtained into an SQL statement, that is, SELECT * FROM user_infoWHERE phone_number = 'xxx', and sends the SQL statement to the database middleware, uses the Druid parser to parse the SQL statement, and converts the SQL statement into an abstract syntax tree (AST) to represent the various components of the SQL statement, such as the SELECT clause, FROM clause, WHERE clause, etc.

[0100] Step S302, traverse each node of the abstract syntax tree, determine the column name node in each node, and match the column name node with the encrypted field to obtain the column field corresponding to the encrypted data.

[0101] The node of the abstract syntax tree refers to a syntax element in the source code, such as an expression, field, statement, function, etc. The column name node refers to the node indicating the column name or column field in each node; for example, in the abstract syntax tree generated by SELECT * FROM user_infoWHERE phone_number = 'xxx', phone_number is the column name node.

[0102] Exemplarily, the server creates a custom visitor class (i.e., Visitor class), which is used to implement the Visitor port provided by the Druid parser; then, the server uses the custom Visitor to traverse the abstract syntax tree. During the traversal process, the Visitor visits each node of the abstract syntax tree and performs corresponding operations according to the type of the node; when the Visitor visits a column name node (such as phone_number in the above example), the Visitor records the column name node. Based on the column name node determined by the Visitor, the server matches the column name node with the encrypted field in the metadata table. If the match is successful, it means that the column name node is an encrypted column field, and the corresponding data thereon is encrypted data, thereby determining the column field corresponding to the encrypted data.

[0103] Step S303, based on the matched column field, determine the information table to which the SQL request corresponding to the column field belongs, push the information table into the stack using the stack structure, obtain the information table name of the SQL statement corresponding to the column field at the top of the stack structure, and determine the column field and the information table to which it belongs.

[0104] Among them, the stack structure refers to a data structure that follows the last-in-first-out principle, that is, the last element added to the stack will be the first element to be removed.

[0105] For example, Visitor uses a stack data structure to maintain context information while traversing the AST. When Visitor encounters the FROM user_info clause, it pushes the table name user_info into the stack. At this point, the top element of the stack is the table name currently being processed. When Visitor accesses the column name node, it obtains the current table name from the top of the stack and determines which information table the column name belongs to. When the relevant operations of the table are processed, the table name is popped from the stack to obtain the column field and the information table to which it belongs.

[0106] In this embodiment, by parsing the SQL statement corresponding to the SQL request, the abstract syntax tree corresponding to the SQL request can be obtained, so that in the process of Visitor traversal, the column name node can be accurately located, and by matching the column name node with the encrypted field, the column field corresponding to the encrypted data in the SQL request can be quickly and accurately determined, thereby facilitating the rewriting of the SQL request in subsequent steps; at the same time, in the process of Visitor traversal, the stack structure is used to maintain context information, which can overcome the defect of the Druid parser lacking context, and the name of the information table can be quickly obtained through the source at the top of the stack to improve the processing efficiency of the encryption process.

[0107] In one embodiment, the types of SQL requests include query requests, insert requests, update requests and / or delete requests.

[0108] The query request is SELECT, which is used to query data from the database. Through SELECT, data of one or more fields in the table can be retrieved, and records that meet the conditions can be filtered out according to specified conditions.

[0109] Insert request, or INSERT, is used to insert new data rows into a table in the database. One or more new records can be added to the specified table.

[0110] An update request, or UPDATE, is used to modify existing records in a table. It can update field values ​​in records that meet the specified conditions based on the specified conditions.

[0111] A delete request, or DELETE, is used to delete records in a table. You can delete records that meet the specified conditions, or delete all records in the table (but retain the table structure).

[0112] In step S202, SQL is rewritten for the column fields and the information table, and the rewritten SQL request includes:

[0113] Obtain a metadata table, and determine the encryption algorithm and encryption algorithm parameters of the column field according to the encrypted field in the metadata table that matches the column field; based on the encryption algorithm, encryption algorithm parameters and the type of SQL request, rewrite the query request, insert request, update request and / or delete request for the column field and the information table to obtain the rewritten SQL request.

[0114] The rewritten SQL request is embedded with an encryption algorithm and encryption algorithm parameters to ensure that when the data of the corresponding storage node is subsequently accessed, the corresponding decryption algorithm can be determined based on the embedded encryption algorithm and encryption algorithm parameters to decrypt the encrypted data.

[0115] In an optional example, the server determines the encryption algorithm and encryption algorithm parameters of the column field based on the encrypted field matching the column field in the metadata table; based on the encryption algorithm, encryption algorithm parameters and the type of SQL request, the server rewrites the query request, insert request, update request and / or delete request for the column field and the information table to obtain the rewritten SQL request. During the rewriting process, different SQL requests are processed separately. The core of SELECT processing is the rewriting of the WHERE condition, and DELETE, INSERT and UPDATE reuse the rewriting of its WHERE condition.

[0116] For example, for a SELECT request, take the request SELECT * FROM user_info WHERE phone_number='xxx' as an example for rewriting. After identifying that the encrypted field to be queried is phone_number, if the encrypted field phone_number is encrypted using the AES algorithm, then there is ENCRYPT('xxx', 'encryption_key'). Based on this, the rewritten SELECT request can be expressed as SELECT * FROM user_info WHERE phone_number=<encrypted_value> ,<encrypted_value> It is the encrypted result, that is, 'xxx' in the aforementioned SELECT request. Usually, the rewritten SELECT request does not directly contain the encrypted value. Instead, it uses parameterized queries or prepared statements to pass the encrypted value as a parameter to ensure secure data transmission.

[0117] For INSERT and UPDATE requests, take the INSERT request as INSERT INTO user_info(phone_number, ...) VALUES ('xxx', ...) or the UPDATE request as UPDATE user_info SETphone_number = 'xxx' WHERE ... as an example for rewriting. After identifying that the encrypted field phone_number needs to be inserted or updated, the encrypted field phone_number is also encrypted using the AES algorithm. Based on this, the rewritten INSERT request can be expressed as INSERT INTO user_info (phone_number, ...) VALUES (<encrypted_value> , ...), the rewritten UPDATE request can be expressed as UPDATE user_info SETphone_number =<encrypted_value> WHERE ....

[0118] For the DELETE request, take the DELETE request as DELETE FROM user_info WHERE phone_number='xxx' as an example for rewriting. After identifying the encrypted field phone_number to be deleted, the encrypted field phone_number is also encrypted using the AES algorithm. Based on this, the rewritten DELETE request can be expressed as DELETE FROMuser_info WHERE phone_number =<encrypted_value> .

[0119] In this embodiment, based on the encryption algorithm, encryption algorithm parameters and the type of SQL request, query requests, insert requests, update requests and / or delete requests are rewritten for column fields and information tables to achieve encryption of encrypted fields. The user is unaware of the encryption process and can seamlessly use the encrypted data without manually performing encryption and decryption operations, which is conducive to improving user experience, enhancing data security and improving data processing efficiency.

[0120] In one embodiment, the middleware also stores shard configuration information.

[0121] The sharding configuration information refers to the detailed settings of parameters such as the sharding algorithm and sharding key involved in performing the sharding operation; the sharding configuration information includes the sharding key and the sharding algorithm. The sharding key is a key field when splitting the database, which determines how the data is distributed to different shards. The sharding algorithm can include but is not limited to the sharding algorithm based on the time range, the hash sharding algorithm, the modulo sharding algorithm, the list sharding algorithm, etc.

[0122] In step S203, the rewritten SQL request is fragmented to obtain a target fragment, and the fragmented SQL request is sent to multiple storage nodes corresponding to the target fragment, including:

[0123] Obtain shard configuration information and initialize shards; extract shard keys in SQL requests based on parsed SQL requests; perform shard calculations on shard keys based on sharding algorithms to obtain shard data, and determine target shards that match shard data based on shard data; send rewritten SQL requests to multiple storage nodes corresponding to the target shards.

[0124] The shard data refers to the data corresponding to each clause in the SQL request being divided into multiple subsets according to a preset sharding algorithm. By comparing these subsets with each preset shard, the target shard corresponding to each subset can be determined.

[0125] Exemplarily, the server obtains the sharding key and sharding algorithm stored in the middleware (i.e., Zookeeper). Taking the hash sharding algorithm as an example, the sharding key is set to user_id, and the server initializes the sharding according to the signed sharding configuration information; when the rewritten SQL request arrives, the server extracts the value of the sharding key user_id, and performs hash calculation on the value of user_id according to the hash sharding algorithm to obtain a hash value (i.e., sharding data). Subsequently, the server calculates which shard (i.e., target shard) the sharding data should be stored on based on the hash value and the preset number of shards (or sharding range). If 4 shards are set, the hash value can be used to determine which shard the data should be stored on by modulo operation (%4); when the server determines the shard where the sharded data should be stored or queried, it sends the rewritten SQL request to the storage node corresponding to the target shard.

[0126] In this embodiment, a sharding algorithm is used to shard the data and then store them on multiple storage nodes to achieve distributed consistency transformation of the storage nodes, thereby ensuring the concurrency and consistency of data processing and ensuring the consistency and synchronization of business data in massive data scenarios.

[0127] In one embodiment, in step S203, the distributed consistency Raft algorithm is used to perform consistency transformation on the storage nodes, so as to send the rewritten SQL request to multiple storage nodes corresponding to the target shard. Figure 4 As shown, step S203 includes the following steps S401 to S404. Among them:

[0128] Step S401, initializing all storage nodes.

[0129] Step S402, selecting a master node for the storage node, and determining other storage nodes except the master node as slave nodes.

[0130] The master node refers to the leader role in the Raft algorithm, which is used to accept client requests, which usually include data writing, reading and other operations, and add the client's request as a new log entry to its own log. At the same time, it sends AppendEntries RPC (remote procedure call) to copy the log entry to other slave nodes in the cluster to maintain the consistency of the cluster and ensure that the logs of all slave nodes are consistent with their own. And, when the log is synchronized to most slave nodes, it instructs the slave node to commit the log.

[0131] The master-slave node refers to the follower role in the Raft algorithm. Under normal circumstances, the slave node will not directly process the client's request, but wait for the master node's instructions, accept and persist the log entries synchronized from the master node, and submit the log after the master node indicates that the log can be submitted. When the slave node does not receive the heartbeat message of the master node within a certain period of time, or receives an election request initiated by other nodes, the slave node will participate in the election of a new master node.

[0132] Exemplarily, after initialization, there is no master node among the storage nodes, that is, all slave nodes can be selected as candidates for the master node. The server controls each storage node to select the master node to determine the master node, and then determines the other storage nodes except the master node as slave nodes.

[0133] Step S403, when the SQL request is sent to the target shard, the master node is controlled to perform data encapsulation to encapsulate the SQL request into a log entry and copy it to the master node, and based on the master node's copy request, each slave node is controlled to copy the log entry to the corresponding slave node.

[0134] Among them, the log entry is the basic unit for each storage node to record operation information, which is used for subsequent consistency operations.

[0135] Exemplarily, when a SQL request is sent to a target shard, the server controls the master node to first receive the result from the shard (i.e., the SQL request), including the shard ID, data range, and possible other metadata; then, the server controls the master node to perform data encapsulation to encapsulate the SQL request into a log entry, which contains all necessary information about the shard result, and controls the master node to send the log entry to all slave nodes via an AppendEntries RPC request, which contains the current log index of the master node, the previous log index, the log entry to be copied, and other contents. After each slave node receives the AppendEntries RPC request, the server controls each slave node to check whether the log entry sent by the master node is consistent with its own log. If the log entry is consistent, or the slave node's log lags behind the master node (i.e., there is a hole), the server controls the slave node to append the log entry to its own log, and controls the slave node to send a response to the master node, indicating that the log entry has been successfully received and appended.

[0136] Step S404, when the number of slave nodes that replicate the log entry meets the preset condition, the master node is controlled to return the replication result to send the rewritten SQL request to each storage node.

[0137] Exemplarily, when the server detects that the master node receives successful responses from the majority (usually more than half) of the slave nodes, it can be determined that the number of slave nodes that replicate the log entry meets the preset conditions. Subsequently, the server controls the master node to mark the log entry as submitted, so that the log entry and all previous log entries remain persistent and consistent; then, the server controls the master node to apply the sharding result (i.e., SQL request) in the submitted log entry to the local state machine to send the rewritten SQL request to each storage node.

[0138] In this embodiment, the master node is responsible for processing client SQL requests and maintaining cluster consistency, and the slave node copies the log entries of the master node to maintain data consistency. The two work together to ensure the consistency and reliability of the distributed system and ensure that data is not lost after an abnormal switch, thereby ensuring the high availability of the system. In addition, the data of all nodes is always consistent, avoiding potential problems caused by data inconsistency.

[0139] In one embodiment, Figure 5 As shown, in step S204, the process of data decryption includes the following steps S501 to S504. Among them:

[0140] Step S501, based on the target shard and the log entries in each storage node, query the business data stored in the storage node, obtain and parse the column part of the MySQL protocol packet, and obtain the column information of each column in the business data.

[0141] The column part of the MySQL protocol packet refers to the part of the query result message that describes the information of each column (field) in the query result set; the column information refers to the information or data corresponding to the column field.

[0142] Exemplarily, the server locates the corresponding storage node according to the target shard, queries the business data stored in the storage node according to the log entries in each storage node, obtains and parses the column part of the MySQL protocol packet, and obtains the column information of each column of each business data in the information table.

[0143] Step S502: Match the fields of the column information with the encrypted fields in the metadata table, and determine the fields of the column information that are successfully matched as the decrypted fields.

[0144] Among them, the decryption field refers to the field corresponding to the encryption field of the previously encrypted data during the decryption process. Usually, the fields referred to by the decryption field and the encryption field are the same field, but they are two states of the field in different stages. For example, for the phone_number field, in the decryption stage, it is called the decryption field, and in the encryption stage, it is called the encryption field.

[0145] Exemplarily, after obtaining the column information, the server matches the fields of the column information with the encrypted fields in the metadata table. If the match is successful, it means that the data corresponding to the column information is encrypted data. Correspondingly, when responding, the encrypted data needs to be decrypted and returned to the client. Therefore, the fields of the successfully matched column information are determined as the decrypted fields.

[0146] Step S503: Determine the encryption algorithm and encryption algorithm parameters of the decryption field according to the encryption field that successfully matches the decryption field, and determine the corresponding decryption algorithm based on the encryption algorithm and encryption algorithm parameters.

[0147] The decryption algorithm refers to an algorithm corresponding to the encryption algorithm and used to decrypt encrypted data.

[0148] Exemplarily, the server determines the encryption algorithm and encryption algorithm parameters of the decryption field based on the encryption field that successfully matches the decryption field, and determines the corresponding decryption algorithm based on the encryption algorithm and the encryption algorithm parameters for decrypting the encrypted data.

[0149] Step S504: decrypt the decryption field based on a decryption algorithm to obtain decrypted data.

[0150] In this embodiment, by decrypting the encrypted data, when the client generates an SQL request, the encrypted stored data can be decrypted and fed back to the user, and the user is unaware of the decryption process.

[0151] In an application example, this embodiment proposes a data transparent encryption method based on database middleware, and its processing of sensitive data is mainly divided into a coordinated processing process of a computing node module and a storage node module.

[0152] The computing node module is responsible for setting up the encrypted table and processing business SQL requests. The configuration of the encrypted table includes the setting of the ciphertext column, the setting of the encryption algorithm, and the allocation of the middleware user rights, and the setting information is stored in the metadata of Zookeeper. SQL request processing mainly includes SQL parsing to obtain the table name and column name of the context, the encryption table configuration and column matching stored in the metadata, the calculation of the routing shards combined with the sharding algorithm, and the pipeline action of SQL rewriting. First, in the SQL parsing process, the Druid parser is used to customize the extended visitor to traverse the abstract syntax tree and identify the column name of each SQL; secondly, the stack data structure is used to maintain the context of the syntax tree to overcome the defect of the Druid parser lacking context, and the name of the SQL table is obtained from the source at the top of the stack; secondly, the encryption configuration information of the metadata table stored in the middleware Zookeeper and the encryption algorithm set by the custom setting are combined to rewrite the SQL for data encryption of specific columns. In SQL rewriting, different statements are processed separately for addition, deletion, modification and query. The core of SELECT processing is the rewriting of the Where condition, and DELETE and UPDATE reuse the rewriting of their Where conditions. The user is unaware of the entire encryption process. Then, based on the sharding key and sharding algorithm, the specific shards of the data distribution are calculated, and the rewritten SQL is sent to the bottom layer. Finally, after the bottom layer data is queried, the ciphertext column is identified by combining the metadata table encryption configuration information and the column information contained in the field of the MySQL protocol package, and the ciphertext column data of the queried data is decrypted and encapsulated into a MySQL protocol package and returned to the client. The user is unaware of the decryption process.

[0153] The storage node module uses the distributed consistency Raft algorithm for distributed consistency transformation. For the DN cluster of storage nodes, the ideas of node election, data synchronization, and data persistence are applied to the MySQL engine layer, and the qualification election check, random sleep, and voting processes are added, which greatly reduces the consumption of communication and achieves the final consistency of distributed scenarios.

[0154] The specific steps of the data transparent encryption method based on database middleware in this example are as follows:

[0155] Step 1: Pre-process the SQL request for parsing. First, during the SQL parsing process, use the Druid parser and the custom extended visitor to traverse the abstract syntax tree and identify the column name of each SQL. Second, use the stack data structure to maintain the context of the syntax tree to overcome the defect of the Druid parser's lack of context, and obtain the name of the SQL table through the source at the top of the stack. Third, combine the metadata table encryption configuration information stored in the middleware Zookeeper and the custom encryption algorithm to rewrite the SQL for data encryption of specific columns. In the SQL rewriting, different statements are processed separately for addition, deletion, modification and query. The core of SELECT processing is the rewriting of the Where condition, and DELETE and UPDATE reuse the rewriting of their Where conditions. The user is unaware of the entire encryption process.

[0156] Step 2: Route calculation of computing node module. Based on the preprocessing of SQL parsing, the specific shards of data distribution are calculated according to the sharding key and sharding algorithm (hash, time range, modulus, etc.) stored in Zookeeper, and the rewritten SQL is sent to the bottom layer.

[0157] Step 3: The storage node module uses the distributed consistency Raft algorithm to perform distributed consistency transformation, applies the ideas of node election, data synchronization, and data persistence to the MySQL engine layer, and adds qualification election checks, random sleep, and voting processes, which greatly reduces communication consumption and achieves eventual consistency in distributed scenarios.

[0158] Step 4: After the route is calculated and the SQL request is issued and the underlying data is queried, the ciphertext column is identified by combining the metadata table encryption configuration information and the column information contained in the field of the MySQL protocol package, and the ciphertext column data of the queried data is decrypted and encapsulated into a MySQL protocol package and returned to the client. The user is unaware of the decryption process.

[0159] Based on the above process, this example realizes data modification and query results without omission or duplication in massive data scenarios, and uses SQL parsing optimization, routing calculation, identification and encryption and decryption of ciphertext columns of the computing module, and distributed consistency transformation of the storage module to complete transparent encryption and fast query of business data, thereby combining the security, concurrency and consistency of data processing. Through comparative experiments, it is proved that the present invention completes data transparent encryption while ensuring the consistency and synchronization of business data in massive data scenarios and without additional hardware costs, and the performance is improved by about 4 times, which further verifies the effectiveness of the present invention in transparent encryption and query optimization of distributed databases in massive scenarios.

[0160] Compared with the existing distributed database that only relies on sharding, the method of this example implements a method of transparent data encryption and query optimization while ensuring that the business data queried in the distributed database in the sharding scenario is consistent and complete.

[0161] In terms of functions, this example uses the SQL parsing optimization and routing calculation of the computing module to complete the functions of ciphertext column identification and data transparent encryption; relying on the routing calculation of the computing node, it can quickly locate the routing information of the data, thereby realizing rapid query of massive data.

[0162] In terms of ensuring the consistency and accuracy of distributed database data, the distributed consistency transformation of the storage module is used to ensure that in the event of a partial failure of the distributed database connection storage node, the distributed consistency cluster disaster recovery can be completed, thereby ensuring the security of business data requests for querying the distributed database.

[0163] In terms of performance, this example is compared with the existing distributed database that relies only on sharding. In the case of a single data query request and a single connection, 100,000 set and get operations are performed on business data with a value size of 1KB and 10KB respectively. The test results are shown in Table 1 (in ms):

[0164] Table 1

[0165]

[0166] The performance test results in Table 1 verify that the performance of the data query optimization solution proposed in this example is significantly better than the solution that does not use SQL parsing optimization of computing nodes and consistency transformation of routing and storage modules, and the speed is increased by about 4 times.

[0167] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0168] Based on the same inventive concept, the embodiment of the present application also provides a data encryption query device for implementing the data encryption query method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more data encryption query device embodiments provided below can refer to the limitations of the data encryption query method above, and will not be repeated here.

[0169] In an exemplary embodiment, Figure 6 As shown, a data encryption query device is provided, including: a request parsing module 601, a request rewriting module 602, a shard storage module 603, a data query module 604 and a data encapsulation module 605. Among them:

[0170] The request parsing module 601 is used to receive an SQL request, parse the SQL request, match the parsed SQL request with the configured metadata table, and obtain the column fields and information table corresponding to the encrypted data carried in the SQL request;

[0171] The request rewriting module 602 is used to rewrite the SQL of the column fields and the information table based on the metadata table, embed the encryption algorithm and encryption algorithm parameters in the metadata table into the SQL request, and obtain the rewritten SQL request;

[0172] The shard storage module 603 is used to perform a sharding operation on the rewritten SQL request to obtain a target shard, and send the sharded SQL request to multiple storage nodes corresponding to the target shard; the multiple storage nodes store business information and its corresponding business data in a distributed manner, and the business data includes encrypted data and unencrypted data;

[0173] The data query module 604 is used to obtain the business information stored in the corresponding storage node according to the rewritten SQL request, and determine the encrypted data in the business data based on the metadata table and the MySQL protocol packet, decrypt the encrypted data, and obtain the decrypted data corresponding to the encrypted data;

[0174] The data encapsulation module 605 is used to encapsulate the decrypted data and the unencrypted data into response data and return it.

[0175] In an optional embodiment, the data encryption query device further includes:

[0176] The data configuration module is used to configure the metadata table and store the metadata table in a middleware.

[0177] In an optional embodiment, the request parsing module 601 is also used to receive an SQL request, obtain an SQL statement in the SQL request, and parse the SQL statement to obtain an abstract syntax tree corresponding to the SQL statement; traverse each node of the abstract syntax tree, determine the column name node in each node, and match the column name node with the encrypted field to obtain the column field corresponding to the encrypted data; based on the matched column field, determine the information table to which the SQL request corresponding to the column field belongs, push the information table into the stack using a stack structure, obtain the information table name of the SQL statement corresponding to the column field at the top of the stack structure, and determine the column field and the information table to which it belongs.

[0178] In an optional embodiment, the request rewriting module 602 is also used to obtain a metadata table, determine the encryption algorithm and encryption algorithm parameters of the column field according to the encrypted field in the metadata table that matches the column field; based on the encryption algorithm, encryption algorithm parameters and the type of SQL request, rewrite the query request, insert request, update request and / or delete request on the column field and the information table to obtain a rewritten SQL request; the rewritten SQL request is embedded with the encryption algorithm and encryption algorithm parameters.

[0179] In an optional embodiment, the shard storage module 603 is also used to obtain shard configuration information and perform shard initialization; the shard configuration information includes a shard key and a shard algorithm; based on the parsed SQL request, the shard key in the SQL request is extracted; based on the shard algorithm, the shard calculation is performed on the shard key to obtain shard data, and a target shard matching the shard data is determined based on the shard data; and the rewritten SQL request is sent to multiple storage nodes corresponding to the target shard.

[0180] In an optional embodiment, the shard storage module 603 is also used to initialize all storage nodes; select a master node for the storage nodes, and determine other storage nodes except the master node as slave nodes; when an SQL request is sent to the target shard, control the master node to perform data encapsulation to encapsulate the SQL request into a log entry and copy it to the master node, and control each slave node to copy the log entry to the corresponding slave node based on the master node's copy request; when the number of slave nodes that copy the log entry meets the preset conditions, control the master node to return the copy result to send the rewritten SQL request to each storage node.

[0181] In an optional embodiment, the data query module 604 is also used to query the business data stored in the storage node based on the target shard and the log entries in each storage node, obtain and parse the column part of the MySQL protocol packet, and obtain the column information of each column in the business data; match the field of the column information with the encrypted field in the metadata table, and determine the field of the column information that successfully matches as the decryption field; determine the encryption algorithm and encryption algorithm parameters of the decryption field based on the encrypted field that successfully matches the decryption field, and determine the corresponding decryption algorithm based on the encryption algorithm and the encryption algorithm parameters; decrypt the decryption field based on the decryption algorithm to obtain decrypted data.

[0182] Each module in the above-mentioned data encryption query device can be implemented in whole or in part by software, hardware and their combination. Each module can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the corresponding operations of each module above.

[0183] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Figure 7 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store encrypted data, encrypted fields, encryption algorithms, encryption algorithm parameters, various fields, information tables, sharding configuration information and other data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a data encryption query method is implemented.

[0184] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0185] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the data encryption query method of the above embodiment is implemented.

[0186] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the data encryption query method of the above embodiment is implemented.

[0187] In one embodiment, a computer program product is provided, including a computer program, which implements the data encryption query method of the above embodiment when executed by a processor.

[0188] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0189] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0190] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0191] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A data encryption query method, characterized in that: The method comprises: Receiving an SQL request, parsing the SQL request, matching the parsed SQL request with a configured metadata table, and obtaining a column field and an information table corresponding to the encrypted data carried in the SQL request; Rewrite the column fields and information table in SQL based on the metadata table, embed the encryption algorithm and encryption algorithm parameters in the metadata table into the SQL request, and obtain a rewritten SQL request; The rewritten SQL request is fragmented to obtain a target fragment, and the fragmented SQL request is sent to a plurality of storage nodes corresponding to the target fragment; the plurality of storage nodes are distributedly stored with business information and corresponding business data, wherein the business data includes encrypted data and unencrypted data; Acquire the business information stored in the corresponding storage node according to the rewritten SQL request, determine the encrypted data in the business data based on the metadata table and the MySQL protocol packet, decrypt the encrypted data, and obtain decrypted data corresponding to the encrypted data; The decrypted data and the unencrypted data are encapsulated into response data and returned.

2. The method according to claim 1, characterized in that The receiving of the SQL request, parsing the SQL request, matching the parsed SQL request with the configured metadata table, and obtaining the column fields and information table corresponding to the encrypted data carried in the SQL request, further includes: A metadata table is configured and stored in a middleware; the metadata table includes encryption fields configured based on encryption data and their corresponding encryption algorithms and encryption algorithm parameters.

3. The method according to claim 2, characterized in that The receiving of the SQL request, parsing the SQL request, matching the parsed SQL request with the configured metadata table, and obtaining the column fields and information table corresponding to the encrypted data carried in the SQL request include: Receive an SQL request, obtain an SQL statement in the SQL request, and parse the SQL statement to obtain an abstract syntax tree corresponding to the SQL statement; Traversing each node of the abstract syntax tree, determining a column name node in each node, and matching the column name node with the encrypted field to obtain a column field corresponding to the encrypted data; Based on the matched column field, determine the information table to which the SQL request corresponding to the column field belongs, push the information table into the stack using the stack structure, obtain the information table name of the SQL statement corresponding to the column field at the top of the stack structure, and determine the column field and the information table to which it belongs.

4. The method according to claim 2, characterized in that: The types of the SQL request include query request, insert request, update request and / or delete request; The step of rewriting the column fields and the information table in SQL based on the metadata table, embedding the encryption algorithm and encryption algorithm parameters in the metadata table into the SQL request, and obtaining the rewritten SQL request includes: Acquire the metadata table, and determine the encryption algorithm and encryption algorithm parameters of the column field according to the encrypted field in the metadata table that matches the column field; Based on the encryption algorithm, the encryption algorithm parameters and the type of the SQL request, the query request, the insert request, the update request and / or the delete request are rewritten for the column field and the information table to obtain a rewritten SQL request; the rewritten SQL request is embedded with the encryption algorithm and the encryption algorithm parameters.

5. The method according to claim 2, characterized in that: The middleware also stores shard configuration information; The step of performing a sharding operation on the rewritten SQL request to obtain a target shard, and sending the sharded SQL request to a plurality of storage nodes corresponding to the target shard, includes: Obtain sharding configuration information and initialize sharding; the sharding configuration information includes sharding key and sharding algorithm; Based on the parsed SQL request, extract the shard key in the SQL request; Performing sharding calculation on the sharding key based on the sharding algorithm to obtain sharding data, and determining a target shard matching the sharding data according to the sharding data; The rewritten SQL request is sent to multiple storage nodes corresponding to the target shard.

6. The method according to claim 5, characterized in that The sending the rewritten SQL request to the multiple storage nodes corresponding to the target shards includes: Initialize all storage nodes; Selecting a master node for the storage node, and determining other storage nodes except the master node as slave nodes; When the SQL request is sent to the target shard, control the master node to perform data encapsulation to encapsulate the SQL request into a log entry and copy it to the master node, and control each slave node to copy the log entry to the corresponding slave node based on the copy request of the master node; When the number of slave nodes that replicate the log entry meets a preset condition, the master node is controlled to return the replication result to send the rewritten SQL request to each storage node.

7. The method according to claim 6, characterized in that The process of acquiring the business information stored in the corresponding storage node according to the rewritten SQL request, determining the encrypted data in the business data based on the metadata table and the MySQL protocol packet, and decrypting the encrypted data to obtain decrypted data corresponding to the encrypted data includes: Based on the target shard and the log entries in each storage node, query the business data stored in the storage node, obtain and parse the column part of the MySQL protocol packet, and obtain the column information of each column in the business data; Matching the fields of the column information with the encrypted fields in the metadata table, and determining the fields of the column information that successfully match as the decrypted fields; Determine the encryption algorithm and encryption algorithm parameters of the decryption field according to the encryption field that successfully matches the decryption field, and determine the corresponding decryption algorithm based on the encryption algorithm and encryption algorithm parameters; The decryption field is decrypted based on the decryption algorithm to obtain decrypted data.

8. A data encryption query device, characterized in that: The device comprises: A request parsing module, used to receive an SQL request, parse the SQL request, match the parsed SQL request with a configured metadata table, and obtain column fields and information tables corresponding to the encrypted data carried in the SQL request; A request rewriting module, used to perform SQL rewriting on the column fields and the information table based on the metadata table, embed the encryption algorithm and encryption algorithm parameters in the metadata table into the SQL request, and obtain a rewritten SQL request; A shard storage module is used to perform a sharding operation on the rewritten SQL request to obtain a target shard, and send the sharded SQL request to a plurality of storage nodes corresponding to the target shard; the plurality of storage nodes store business information and its corresponding business data in a distributed manner, and the business data includes encrypted data and unencrypted data; A data query module, used to obtain the business information stored in the corresponding storage node according to the rewritten SQL request, and determine the encrypted data in the business data based on the metadata table and the MySQL protocol packet, decrypt the encrypted data, and obtain decrypted data corresponding to the encrypted data; The data encapsulation module is used to encapsulate the decrypted data and the unencrypted data into response data and return the response data.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images