Data query method, data storage method, device and computer equipment

Abstract

The present application discloses a data query and storage method, device, equipment, storage medium and program product, the method comprising: encrypting the query field according to a preset encryption rule to obtain a query key; querying in a first database according to the query key to obtain a query value corresponding to the query key; querying in a second database according to the query value for first type data corresponding to the query value to obtain a query result corresponding to the query field. The present application realizes data query by converting the mapping relationship between the query field and the first type of data into a mapping relationship between the query key and the query value after the data and its field are encrypted, which can effectively ensure privacy during the query process, and improves data query efficiency because all data does not need to be decrypted in the database.

Description

Technical Field

[0001] This application relates to the field of data processing technology, specifically to a data query and data storage method, apparatus, and computer equipment. Background Technology

[0002] Information anonymization refers to the process of modifying and storing data according to specific anonymization rules to protect sensitive data, such as privacy information. However, during the query process, because the anonymized data stored in the database often differs from the user's input, data processing is often required to obtain accurate results. Therefore, the data query performance of related technologies is not ideal. Summary of the Invention

[0003] To address the aforementioned technical problems, embodiments of this application provide a data query and data storage method, apparatus, and computer device, aiming to solve the problem of unsatisfactory data query results.

[0004] In a first aspect, embodiments of this application provide a data query method, including:

[0005] The query fields are encrypted according to the preset encryption rules to obtain the query key;

[0006] A query is performed in the first database based on the query key to obtain the query value corresponding to the query key;

[0007] Based on the query value, query the first type of data corresponding to the query value in the second database to obtain the query result corresponding to the query field. The second database is used to store multiple first type of data, and the first database is used to store the query value and query key corresponding to each first type of data.

[0008] Secondly, embodiments of this application provide a data storage method, including:

[0009] Split the first type of data in the second database to obtain the first type of fields;

[0010] The first type of data and the first type of field are encrypted according to a preset encryption rule to obtain a first encrypted value of the first type of data and a second encrypted value of the first type of field.

[0011] The first encrypted value is used as the query value and the second encrypted value is used as the query key, and they are stored together in the first database.

[0012] Thirdly, embodiments of this application provide a data query device, including:

[0013] The encryption module is used to encrypt the query fields according to preset encryption rules to obtain the query key;

[0014] The query module is used to perform a query in the first database based on the query key and obtain the query value corresponding to the query key;

[0015] The query module is further configured to query the first type of data corresponding to the query value in the second database based on the query value, and obtain the query result corresponding to the query field. The second database is used to store multiple first type of data, and the first database is used to store the query value and query key corresponding to each first type of data.

[0016] Fourthly, embodiments of this application provide a data storage device, including:

[0017] The splitting module is used to split the first type of data stored in the second database to obtain the first type of fields;

[0018] An encryption module is used to encrypt the first type of data and the first type of field according to a preset encryption rule to obtain a first encrypted value of the first type of data and a second encrypted value of the first type of field;

[0019] The storage module is used to store the first encrypted value as a query value and the second encrypted value as a query key in a first database.

[0020] Fifthly, embodiments of this application also provide a computer device, including a processor and a memory, wherein the memory stores a plurality of instructions; the processor loads instructions from the memory to execute the steps of the data query method or the data storage method as described above.

[0021] Sixthly, embodiments of this application also provide a computer-readable storage medium storing a plurality of instructions adapted for loading by a processor to execute the steps of the data query method or the data storage method as described above.

[0022] In a seventh aspect, embodiments of this application also provide a computer program product, the computer program product including a computer program, the computer program being executed by a processor using the steps of the data query method or the data storage method described above.

[0023] The method provided in this application embodiment stores multiple types of first-type data and their query values ​​and query keys separately in a first database and a second database. This achieves isolation between the original data of the first-type data and the parameters used for querying, such as query keys and query values. Due to this isolation, it is impossible to query and view the first-type data without prior knowledge of the preset encryption rules and the correspondence between the first database and the second database, thus ensuring the security of the first-type data.

[0024] When querying type 1 data, the query field is encrypted using a preset encryption rule to obtain the query key. The corresponding query value is then quickly retrieved in the first database. This query value can then be used to retrieve the corresponding type 1 data in the second database, serving as the query result for the query field. By storing multiple types 1 data and their corresponding query keys in different databases, along with the mapping relationship between query values ​​and query keys, the data query process is transformed from relying on the mapping relationship between fields and data to a mapping relationship between data and its encrypted query keys and query values. This effectively ensures privacy during the query process while improving data query efficiency because it eliminates the need to decrypt all data in the database. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a flowchart illustrating the steps of a data query method provided in an embodiment of this application.

[0027] Figure 2 This application provides a schematic flowchart illustrating the steps for querying fragmented storage in an embodiment of the present application.

[0028] Figure 3 This is a flowchart illustrating the steps for constructing a cache filter to improve data query efficiency, as provided in an embodiment of this application.

[0029] Figure 4 This application provides a schematic flowchart illustrating the steps of querying a second database based on a query value, as illustrated in an embodiment of the present application.

[0030] Figure 5 This application provides a schematic flowchart illustrating a step for querying based on a data identifier in an embodiment of the present application.

[0031] Figure 6 This is a flowchart illustrating the steps of a data storage method provided in an embodiment of this application.

[0032] Figure 7 A schematic diagram of a data structure for query keys and query values ​​stored in a database, provided for an embodiment of this application;

[0033] Figure 8 This is a schematic diagram of the structure of a data query device provided in an embodiment of this application;

[0034] Figure 9 This is a schematic diagram of the structure of a data storage device provided in an embodiment of this application;

[0035] Figure 10 This is a schematic diagram of the internal structure of the computer device provided in the embodiments of this application. Detailed Implementation

[0036] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. At the same time, in the description of the embodiments of this application, the terms "first," "second," etc., are only used to distinguish descriptions and should not be construed as indicating or implying relative importance. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0037] To better understand the data query and storage method, apparatus, device, storage medium, and program product provided in this application, the relevant application scenarios of this application will first be described. It should be noted that the following is merely an example illustrating one applicable scenario; the method provided in this application can be applied to any scenario involving encrypted data storage and querying.

[0038] Optionally, the data query and data storage methods provided in this application are mainly applied to scenarios where sensitive user data needs to be anonymized and stored in a database for querying. For example, a common approach is to create a user table in MySQL (a relational database) to store sensitive user data, such as user ID (identity number), user name, ID card number, mobile phone number, etc. Of course, sensitive data can also typically include other information, such as address, place of origin, hobbies, birthday, etc. This data can usually be stored in other sub-tables and associated with the user ID in the user table. In this case, to avoid privacy leaks, the database typically uses preset anonymization rules, such as the AES (Advanced Encryption Standard) encryption algorithm, to encrypt and transform the data, such as mobile phone numbers and ID card numbers. However, in subsequent query processes, a corresponding query process usually needs to be designed to ensure the normal operation of the query process.

[0039] Currently, common methods for de-identifying data querying include database-level decryption queries and queries based on constructing word segmentation-ciphertext mapping tables. The former involves inputting decryption commands into the query statement, requiring the database to filter data that, after decryption, meets the requirements of the query fields. For example, the database might be required to decrypt the data using the AES algorithm based on the key provided in the query statement, then filter out users whose phone numbers contain "1234". This method is low-cost and relatively easy to implement, but in practice, it suffers from low query speed due to the inability to optimize queries through data indexing. Furthermore, the database may not be able to guarantee that it processes the data using the same encryption / decryption algorithm as the program, resulting in the inability to decrypt and complete the query within the database. The latter method, on the other hand, has rigid requirements on the granularity of the query segments, which becomes fixed after the data is stored in the database. This makes it inflexible, especially in fuzzy queries where the de-identified fields input by the user are not fixed. This often requires allocating more word segmentation-ciphertext mapping tables, leading to the substitution of numerous logical tables or fields, impacting database query efficiency. Therefore, existing de-identified data query methods, especially fuzzy queries, suffer from unsatisfactory query results to varying degrees, necessitating optimization of data structures and query algorithms.

[0040] To address the aforementioned issues, this application provides a data query and storage method, apparatus, device, storage medium, and program product. Data querying is achieved by transforming the mapping relationship between fields and data into a mapping relationship between encrypted query keys and query values. This effectively ensures privacy during the query process and improves query efficiency by eliminating the need to decrypt all data in the database. For details, please refer to... Figure 1, Figure 1 This is a flowchart illustrating the steps of a data query method provided in an embodiment of this application, specifically including steps S110 to S130:

[0041] S110, encrypt the query field according to the preset encryption rules to obtain the query key.

[0042] In this embodiment, by processing the query field according to a pre-defined encryption rule, an encrypted query field can be obtained. This ensures data privacy throughout the entire query process when using the encrypted query field to complete the data query. Specifically, the pre-defined encryption rule is typically a fixed encryption rule agreed upon in advance by both parties possessing data storage and requiring query permissions. Of course, depending on actual needs, the pre-defined encryption rule can also be a dynamically changing encryption method, etc. For example, the pre-defined encryption rule can be implemented using a hash algorithm (which transforms an input of arbitrary length into an output of fixed length through a hash algorithm) or an MD5 (Message-Digest Algorithm 5, a cryptographic hash function) algorithm. Of course, using other algorithms that can encrypt the query field is also feasible. This embodiment does not limit the encryption process.

[0043] Furthermore, in order to retrieve the de-identified data related to the query field from the second database, i.e., the first type of data mentioned below, in this embodiment, the obtained encrypted value will be used as a query key to further query the corresponding query value in the first database. For details, please refer to the following steps and their explanations.

[0044] S120, perform a query in the first database according to the query key to obtain the query value corresponding to the query key.

[0045] In this embodiment, the first database refers to a database that pre-stores query values ​​corresponding to the encrypted query fields, which can be used to retrieve the desired query results in another database. In other words, by using the encrypted query fields, i.e., the query keys, to perform a query in the first database, a certain query value can be retrieved, and this query value can be further used to retrieve the actual query results in another database.

[0046] Specifically, the first database typically refers to a non-relational database formed through key-value pairs, also known as a key-value database, which stores data using keys and values. For ease of description, in the following specific embodiments, a key-value database will be used as the first database for illustration.

[0047] Furthermore, for a given query key, the query value associated with that key can be quickly and accurately retrieved in the key-value database. This query value can include multiple values, each associated with different sensitive data. To ensure that the obtained query value can be used to retrieve the corresponding sensitive data in the second database, the query value stored in the key-value database is typically associated with the sensitive data in the second database. That is, the first database stores the query value and query key corresponding to each piece of anonymized data. For example, the query value and query key can be obtained by processing the sensitive data and its fields in the second database according to certain rules, or the query value and query key can be associated with the sensitive data and its fields in the second database, such as a user ID.

[0048] It is understandable that, in order to ensure the effectiveness of data queries, it is usually necessary to appropriately process the sensitive data in the second database and select it as the query value for the appropriate query key. Therefore, based on the query method provided in the embodiments of this application, the embodiments of this application further provide an implementation scheme for how to store data in association in a key-value database. Specifically, for ease of description, it will be combined with the following... Figures 6-10 A detailed explanation will be provided.

[0049] Furthermore, as an embodiment of another data query method provided in this application, considering that there are many sensitive data types stored in the database, such as user names, mobile phone numbers, ID card numbers, etc., as mentioned above, and that users usually need to query specific types of query fields when they need to query, such as querying user information whose mobile phone number contains the field "1234" or whose ID card number contains the field "123456", in order to further improve query efficiency, during the construction of the first database, considering the differences between different field types, different sharded databases can be constructed for different field types. That is, the first database usually includes multiple databases. In this case, during the process of querying the first database according to the query key and obtaining at least one query value corresponding to the query key, it is often necessary to first determine the corresponding third database from multiple first databases according to the field type of the query field, and then use the query key to query in this third database to obtain the corresponding query value. For details, please refer to [link to relevant documentation]. Figure 2 , Figure 2 This application provides a flowchart illustrating the steps for querying fragmented storage, specifically including steps S210 to S220:

[0050] S210, determine a third database from multiple first databases based on the field type of the query field.

[0051] In this embodiment, the field type of the query field typically refers to header information such as "mobile phone number," "ID card number," or "user name," and may also include header information such as "address" or "place of origin." This embodiment does not elaborate on the field type of the query field; any header information in the main table or sub-tables stored in the second database can be considered as the field type of the query field provided in this application.

[0052] Correspondingly, the first database can also include multiple databases, each corresponding to different field types. For example, the first database may include a "mobile phone number segmentation database", an "ID card number segmentation database", etc., which will not be elaborated further in this embodiment.

[0053] Furthermore, in this embodiment, by parsing and determining the field type of the query field, a first target database corresponding to the field type of the query field can be determined from multiple first databases. For example, when it is identified that a query needs to be performed on mobile phone numbers containing "1234", the "mobile phone number shard database" can be used as the first target database for subsequent queries.

[0054] S220, perform a query in the third database according to the query key to obtain the query value corresponding to the query key.

[0055] By identifying a third database related to the field type of the query field and performing a query in the third database based on that query key, query efficiency can be effectively improved while accurately retrieving the query value related to the anonymized database in the second database from the third database.

[0056] Furthermore, as another embodiment of the data query method provided in this application, in order to further improve data query efficiency and quickly determine whether the second database contains query results for the query field, specifically, the query keys in the first database, i.e., the key-value database, can be statistically analyzed, and a cache filter can be designed. In this case, the cache filter can be used to quickly determine whether the key-value database contains the corresponding query key, thereby further determining whether the second database contains query results for the query field. For details, please refer to... Figure 3 , Figure 3 The following is a detailed flowchart illustrating the steps involved in constructing a cache filter to improve data query efficiency, as provided in an embodiment of this application.

[0057] In this embodiment of the application, by introducing a cache filter before the step of querying the first database using the query key value, the data query efficiency can be further improved to quickly determine whether the first database has a corresponding query key. Specifically, the steps include the following:

[0058] S310: Query the cache filter according to the query key value, and determine whether the query key exists in the cache filter. If yes, proceed to step S120; otherwise, proceed to step S320.

[0059] In this embodiment, a cache filter refers to a filter that can quickly identify whether an element is in a set. For example, a Bloom filter is commonly used. This embodiment does not limit the specific structure of the cache filter. Any filter that can quickly and relatively accurately identify whether an element is in a set is feasible.

[0060] In this embodiment of the application, when it is determined that a query key exists in the cache filter, it can be assumed that there is a high probability that the query key exists in the first database. At this time, the aforementioned step S120 can be executed, that is, a query is performed in the first database according to the query key, to obtain at least one query value corresponding to the query key.

[0061] S320 returned no query data.

[0062] In this embodiment of the application, corresponding to the above, when it is determined that there is no query key in the cache filter, it is assumed that the query key definitely does not exist in the first database. At this time, no query data can be returned directly to the user to prompt the user that the corresponding data cannot be found based on the query field.

[0063] By using the above methods, a large number of query fields that do not have corresponding data can be effectively filtered out, thus providing a certain degree of protection against abnormal attacks.

[0064] S130, based on the query value, query the first type of data corresponding to the query value in the second database to obtain the query result corresponding to the query field.

[0065] As described above, the query values ​​obtained from the query in the first database can be further used to query the actual query results in another database. Therefore, in this embodiment, the second database typically stores the query values ​​from the first database and the associated data pairs of the query results corresponding to each query value. This second database is usually a relational database, such as a MySQL database, to store multiple first-type data and their corresponding query values. For ease of description, a relational database will be used as the second database in subsequent embodiments.

[0066] After obtaining the query value associated with the sensitive data in the second database by querying the first database, which is the key-value database, the sensitive data obtained by querying the second database based on the query value can be used as the query result of the query field.

[0067] For example, as a common and feasible implementation, the query value itself can be sensitive data in the relational database. That is, the key-value database uses a sensitive field as the query key and directly uses sensitive data containing that sensitive field as the query value. For instance, for the sensitive data "3456", the sensitive data "3456" can be associated with the encrypted result of a sensitive field within "3456", such as field "45", and stored as a key-value pair. Then, when the user inputs the query field "45", and the query key is obtained through encryption, a query in the key-value database will directly return the sensitive data "3456" to the user. Of course, to ensure the query results are accurate, the sensitive data "3456" can also typically be associated with the encrypted results of sensitive fields "34" and "56", stored as key-value pairs.

[0068] Of course, the above only provides one feasible implementation scheme. It is understandable that directly storing sensitive data in a key-value database for querying would increase the risk of data leakage, thereby affecting user data privacy. Therefore, as another feasible implementation scheme, the query value can also be desensitized by appropriately processing the sensitive data stored in the relational database. For example, similar to the aforementioned method of encrypting the query field to obtain the query key, the query value is obtained by encrypting the sensitive data stored in the relational database. That is, the query value stored in the key-value database is the encrypted value of the sensitive data stored in the relational database. In this case, in order to be able to query the relational database based on the query value to obtain the corresponding sensitive data, the query value can also be associated with the identification information of the sensitive data stored in the relational database. For example, it can be associated with the index set to obtain the data identifier used to indicate the sensitive data in the relational database. Then, the query of the relational database can be completed based on the data identifier to obtain the corresponding sensitive data as the query result corresponding to the query field.

[0069] For details, please refer to Figure 4 , Figure 4 This application provides a flowchart illustrating a step for querying a second database based on a query value, specifically including steps S410 to S420:

[0070] S410, determine the data identifier associated with the query value.

[0071] In this embodiment, by associating the query value obtained after encrypting or otherwise processing sensitive data with a data identifier used to indicate sensitive data in the relational database, the data identifier associated with the query value can be determined. For example, the association between the query value and the data identifier can be pre-stored in an index set. Of course, directly recording the mapping relationship between the query value and the data identifier is also feasible, and this embodiment does not limit this. Then, after obtaining the query value using a key-value database, the corresponding data identifier can be further determined based on the query value. Specifically, the data identifier here can be understood as a unique identifier used to indicate sensitive data in the relational database, such as the aforementioned user ID. Of course, using other information as the data identifier is also feasible, and this embodiment will not elaborate further.

[0072] For a clearer understanding, please refer to Table 1 below. Table 1 shows a schematic diagram illustrating the effect of an index set, as follows:

[0073] The encrypted value (hash) of sensitive data. Data Identifier (User ID) hash_1 1 hash_2 2 hash_3 3 hash_4 4 hash_5 5

[0074] As can be seen, the index set specifically includes two columns. The left side stores the encrypted values ​​of sensitive data, such as the encrypted values ​​of ID card numbers and mobile phone numbers, while the right side stores the corresponding user IDs. At this time, when the corresponding query value is obtained from the key-value database, it is compared with the encrypted values ​​of sensitive data stored on the left side to find the data identifier associated with the query value, which can be used to retrieve the corresponding sensitive data in the relational database later.

[0075] S420: Based on the data identifier, query the first type of data corresponding to the query value in the second database to obtain the query result corresponding to the query field.

[0076] In this embodiment of the application, the data identifier is further used to perform a query in the second database, that is, the relational database. Since the data identifier can be understood as the unique identifier of sensitive data in the relational database, the corresponding sensitive data can also be retrieved and the sensitive data can be output as the query result corresponding to the query field.

[0077] Of course, using data identifiers to query relational databases can further improve data query efficiency. Understandably, users typically need to input query statements similar to SQL (Structured Query Language) to query relational databases. These queries usually include query fields. For example, a query like "Please filter users whose mobile phone numbers contain '1234'" can usually be parsed to obtain the corresponding query fields. Then, a query is performed in a second database based on the data identifier, and the retrieved sensitive data is used as the query result corresponding to the query fields. Alternatively, the query fields in the query statement can be replaced with data identifiers to obtain an updated query statement, allowing for faster completion of the relational database query. For details, please refer to [link to relevant documentation]. Figure 5 , Figure 5 This application provides a flowchart illustrating a step for querying based on a data identifier, specifically including steps S510 to S520:

[0078] S510, the data identifier is used to replace the query field in the query statement to obtain the updated query statement.

[0079] In this embodiment of the application, when the query fields obtained from parsing the query statement include multiple fields, in order to retrieve the sensitive data that meets the requirements of the query statement from the database, it is usually possible to directly replace the query fields in the query statement with data identifiers to obtain an updated query statement. Specifically, for ease of understanding, taking the query statement "Please filter out users whose mobile phone numbers contain '1234' and whose ID card numbers contain '123456'" as an example, through the aforementioned steps, the user data identifiers corresponding to the sensitive field '1234' in the mobile phone number are determined to include users A, B, and C, while the user data identifiers corresponding to the sensitive field '123456' in the ID card number are determined to include users C, D, and E. At this time, by replacing the corresponding sensitive fields with the aforementioned user data identifiers, a query statement similar to "Please filter out users who are in the set 'user A, user B, user C' and in the set 'user C, user D, user E'" can be obtained.

[0080] S520: Based on the updated query statement, perform a query in the second database and use the first type of data obtained from the query as the query result.

[0081] In this embodiment, by using the updated query statement to query the second database, the corresponding sensitive data can be directly obtained from the second database as the query result. Furthermore, it can be seen that by introducing a user identifier, query statements, such as SQL statements, can be used to quickly retrieve data.

[0082] The method provided in this application stores multiple types of first-type data and their query values ​​and query keys separately in a first database and a second database. This isolates the original data of the first-type data from the parameters used for querying, such as query keys and query values. Due to this isolation, the first-type data cannot be queried or viewed without prior knowledge of the preset encryption rules and the correspondence between the first and second databases, thus ensuring the security of the first-type data. When it is necessary to query the first-type data, the query field is encrypted using preset encryption rules to obtain the query key, and the corresponding query value is quickly retrieved in the first database. Then, the corresponding first-type data can be retrieved in the second database using this query value as the query result corresponding to the query field. By using different databases to store multiple types of first-type data and their corresponding query keys, as well as the mapping relationship between query values ​​and query keys, the data querying method, which previously relied on the mapping relationship between fields and data, is now transformed into a mapping relationship between data and its encrypted query keys and query values. This effectively ensures privacy during the query process and improves data query efficiency because it eliminates the need to decrypt all data in the database.

[0083] Based on the aforementioned data query method, this application further provides a corresponding data storage method. For details, please refer to [link / reference needed]. Figure 6 , Figure 6 This application provides a schematic flowchart of a data storage method, specifically including steps S610 to S630:

[0084] S610, split the first type of data in the second database to obtain the first type of field.

[0085] In this embodiment, by splitting the first type of data (i.e., sensitive data) stored in the second database in a certain way, several first type fields (i.e., sensitive fields) can be obtained. For example, taking the mobile phone number "123456789" as an example, based on the conventional splitting method, it can be split into sensitive fields such as "123", "456", and "789". Of course, in some cases, the sensitive data can also be split into sensitive fields such as "1234", "56789" or "12", "3456", etc. This application does not limit the specific form of splitting.

[0086] S620, the first type of data and the first type of field are encrypted according to a preset encryption rule to obtain a first encrypted value of the first type of data and a second encrypted value of the first type of field.

[0087] In this embodiment, in conjunction with the aforementioned step S110, the preset encryption rule is typically a fixed encryption rule agreed upon in advance by both parties possessing data storage and requiring query permissions. Of course, based on actual needs, the preset encryption rule can also be a dynamically changing encryption method, etc. Therefore, the preset encryption rule here is usually similar to that in step S110, and can be implemented by using a hash algorithm (which transforms an input of arbitrary length into an output of fixed length through a hash algorithm) or an MD5 (Message-Digest Algorithm 5, a cryptographic hash function) algorithm to encrypt the fields of the first type of data and the second type of data. Of course, it is also feasible to use other algorithms that can encrypt the query field. This embodiment does not limit the encryption process here.

[0088] S630, the first encrypted value is used as a query value and the second encrypted value is used as a query key, and they are associated and stored in the first database.

[0089] In this embodiment of the application, the first encrypted value and the second encrypted value are used as query value and query key, respectively, and stored in the first database in the form of key-value pairs. Subsequently, when the sensitive field input by the user is encrypted, the second encrypted value containing the sensitive data of this sensitive field can be read from the first database according to the corresponding encrypted value as the query key.

[0090] Of course, it should be noted that in the process of storing data in the first database, that is, the key-value database, it is also possible to store sensitive data in the corresponding sharded cache database based on the data type of the data.

[0091] Furthermore, in the process of storing the first and second encrypted values ​​as key-value pairs in the first database, considering the characteristics of the first and second encrypted values, this application also provides a key-value pair storage format. Specifically, to more flexibly adapt to different granularity field sizes input by the user—for example, users may have different ways of splitting mobile phone numbers or ID card numbers—multiple second encrypted values ​​obtained by processing different sensitive fields can typically be recorded in a query array of the first database. Specifically, the query array length can typically be initialized to 16, and when the data volume is large, it can be easily expanded by 16 storage locations on the original array, thereby adaptively meeting the different granularity field sizes input by the user. Based on this, considering that different sensitive data may correspond to the same sensitive field, for example, sensitive data "123456789" and sensitive data "456789123" both have the same sensitive field "123", meaning that the same key may correspond to multiple different values, the multiple first encrypted values ​​obtained by processing different sensitive data can be written into a linked list structure associated with the query key, based on a linked list structure. That is, the first encrypted value and the second encrypted value are used as the query value and query key, respectively, and stored together in the first database, specifically including:

[0092] Write the second encrypted value as the query key into the query array of the first database;

[0093] Write the first encrypted value as the query value into the linked list structure associated with the query key.

[0094] At this point, when writing the second encrypted value as a query key into the query array of the first database, it is first determined whether there is a query key corresponding to the second encrypted value in the query array of the first database. If there is no query key corresponding to the second encrypted value in the query array, the second encrypted value will be written into the query array of the first database as a query key. If there is already a query key corresponding to the second encrypted value in the query data, the first encrypted value of the sensitive data will be directly written into the linked list structure associated with this query key as the query value.

[0095] Furthermore, to further improve data query efficiency, the data structures associated with different query keys for storing query values ​​will change based on the number of stored query values. Specifically, it can be determined whether the number of query values ​​included in the first database, i.e., the linked list structure, exceeds a preset threshold. If the number of query values ​​included in the linked list structure exceeds the preset threshold, the linked list structure associated with the query key can be adjusted to a tree structure, and the first encrypted value can be written as the query value into this tree structure. For example, as a feasible implementation, the preset threshold can be set to 8. That is, when the number of sensitive data containing a certain sensitive field is less than 8, the first encrypted value of these sensitive data will be stored in a linked structure associated with the second encrypted value of the sensitive field. When the number of sensitive data containing a certain sensitive field is more than 8, the first encrypted value of these sensitive data will be stored in a tree structure associated with the second encrypted value of the sensitive field.

[0096] For details, please refer to [link / reference]. Figure 7 , Figure 7 This is a schematic diagram of a data structure for query keys and query values ​​stored in a database, provided as an embodiment of this application.

[0097] Building upon the aforementioned method of storing the first encrypted value of sensitive data and the second encrypted value of sensitive fields in key-value pairs in the first database, it is further possible to associate the first encrypted value and the data identifier of sensitive data in an index set. This allows the data identifier of sensitive data to be retrieved based on the index set after obtaining the corresponding query value through the query key, thereby completing the query for sensitive data within the second database and serving as the query result for the query field.

[0098] The method provided in this application splits sensitive data to obtain sensitive fields, encrypts the sensitive data and sensitive fields, and records them in a first database as key-value pairs. This facilitates subsequent queries on a query field by retrieving the encrypted value of the sensitive field containing that query field, which can then be used to determine the sensitive field from a second database. Since the first database can store multiple different query keys corresponding to different query fields, as well as query values ​​related to sensitive data corresponding to those query keys, it also ensures that users can perform fuzzy searches using any query field, improving the flexibility of data retrieval.

[0099] To clearly understand the complete implementation process of the data query method and data storage method provided in the embodiments of this application, the following will be combined with the foregoing Figures 1 to 7 The provided content offers a complete step-by-step process for a specific implementation scheme of data storage and querying, specifically including the following steps:

[0100] (1) Create a sys_user table in MySQL to store the core information of users and perform table partitioning. The sys_user table only stores core data such as user ID, username, phone number, and ID card number. Non-core data is divided into multiple sub-tables, such as sys_user_extend_1, sys_user_extend_2, and sys_user_extend_n, to store other information such as home address, place of origin, and personal hobbies. The id of the sys_user table is used as the association id to de-identify information such as username, phone number, ID card number, and email address. A full-text index is created to speed up fuzzy search and also supports pagination operations well.

[0101] (2) Regarding the storage structure and information caching of sensitive data, firstly, the data in the sys_user table is sharded according to different data types, and different data types are stored in different shards to prevent the data volume from becoming too large if stored in only one allocation. For example, there are shards for phone, email, username, and ID number. In each shard, the data storage structure is an upper-level array and a lower-level hybrid structure of linked lists and trees.

[0102] (3) For sensitive data in the sys_user table, after determining a suitable shard, it is split to obtain multiple sensitive fields. Then, these sensitive fields and sensitive data are encrypted using hash or MD5 to obtain encrypted values. The encrypted values ​​of the sensitive fields are used as query keys, and the encrypted values ​​of the sensitive data are used as query values ​​and stored in the corresponding shard.

[0103] (4) During the storage process, first determine whether the upper array stores the query key. If it does not exist, record the query key in the upper data and record the corresponding query value in the linked list structure of the query key. If it exists, directly record the corresponding query value in the linked list structure of the query key. The array is initialized with a length of 16. When the storage space of the array is insufficient, the original array can be expanded by 16 storage locations.

[0104] (5) When recording the corresponding query value in the linked list structure of the query key, it is determined whether the query value is stored in the linked list structure of the query key. If it is, it is not stored; otherwise, it is stored in the linked list structure of the query key. If the linked list is too long, for example, more than 8 nodes, the linked list is treeified to reduce its length and improve query efficiency.

[0105] (6) After processing each sensitive data in the second database, the data can be stored in the cache as a whole. At the same time, a cache filter is built using all the query keys contained in the upper layer data, and the encrypted value of the sensitive data and the user identifier corresponding to the sensitive data, i.e., the user ID, are associated and stored in the index set.

[0106] (7) When data needs to be queried, after hashing or MD5 encrypting the query field, first query the cache filter. If the cache filter finds the relevant data field, it means that there is data in the system and the query can be performed. If the cache filter does not find the relevant data field, it means that there is no relevant data in the system and no data is returned directly.

[0107] (8) Using the encrypted value of the query field as the query key, perform a query in the cache. Read the query value under this query key, organize the obtained data to obtain a dataset, and then query the index set to retrieve identifier data, such as user IDs. This allows for batch querying of the `sys_user` data in the second database, i.e., the MySQL database. The obtained data is the data retrieved through fuzzy query, and all sensitive data in the retrieved data is encrypted, displaying encrypted characters externally. If partial display is required, partial desensitization can be performed.

[0108] To better implement the data query method and data storage method provided in the embodiments of this application, this embodiment also provides a data query device, building upon the data query method provided in the embodiments of this application. For example, as... Figure 8 As shown, the data query device may include:

[0109] The encryption module 810 is used to encrypt the query field according to the preset encryption rules to obtain the query key;

[0110] The query module 820 is used to perform a query in the first database based on the query key and obtain the query value corresponding to the query key.

[0111] The query module 820 is further configured to query the first type of data corresponding to the query value in the second database according to the query value, and obtain the query result corresponding to the query field. The second database is used to store multiple first type of data, and the first database is used to store the query value and query key corresponding to each first type of data.

[0112] In some embodiments, the query module 820 is further configured to determine a third database from a plurality of first databases based on the field type of the query field; and to perform a query in the third database based on the query key to obtain the query value corresponding to the query key.

[0113] In some embodiments, the query module 820 is further configured to determine a data identifier associated with the query value; perform a query in a second database based on the data identifier, query the first type of data corresponding to the query value, and obtain the query result corresponding to the query field.

[0114] In some embodiments, the query module 820 is further configured to replace the query field in the query statement with the data identifier to obtain an updated query statement; perform a query in the second database according to the updated query statement, and use the first type of data obtained from the query as the query result.

[0115] In some embodiments, the query module 820 is further configured to perform a query in the cache filter based on the query key; if the query key exists in the cache filter, perform the step of querying the first database based on the query key to obtain the query value corresponding to the query key; if the query key does not exist in the cache filter, return no query data.

[0116] The data query device provided in this application embodiment stores multiple types of first-type data and their query values ​​and query keys separately in a first database and a second database. This achieves isolation between the original data of the first-type data and the parameters used for querying, such as query keys and query values. Due to this isolation, the first-type data cannot be queried or viewed without prior knowledge of the preset encryption rules and the correspondence between the first and second databases, thus ensuring the security of the first-type data. When it is necessary to query the first-type data, the query field is encrypted using preset encryption rules to obtain the query key, and the corresponding query value is quickly retrieved in the first database. Then, the corresponding first-type data can be retrieved in the second database using this query value as the query result corresponding to the query field. By using different databases to store multiple types of first-type data and their corresponding query keys, as well as the mapping relationship between query values ​​and query keys, the data querying method, which previously relied on the mapping relationship between fields and data, is now transformed into a mapping relationship between data and its encrypted fields' query keys and query values. This effectively ensures privacy during the query process and improves data query efficiency because it eliminates the need to decrypt all data in the database.

[0117] This application also provides a data storage device, for example, such as... Figure 9 As shown, the data storage device may include:

[0118] The splitting module 910 is used to split the first type of data stored in the second database to obtain the first type of field;

[0119] The encryption module 810 is also used to encrypt the first type of data and the first type of field according to a preset encryption rule to obtain a first encrypted value of the first type of data and a second encrypted value of the first type of field.

[0120] The storage module 920 is used to store the first encrypted value as a query value and the second encrypted value as a query key in a first database.

[0121] In some embodiments, the storage module 920 is further configured to write the second encrypted value as a query key into the query array of the first database; and to write the first encrypted value as a query value into the linked list structure associated with the query key.

[0122] In some embodiments, the storage module 920 is further configured to determine whether there is a query key corresponding to the second encrypted value in the query array of the first database; if there is no query key corresponding to the second encrypted value in the query array, execute the step of writing the second encrypted value as a query key into the query array of the first database; if there is a query key corresponding to the second encrypted value in the query array, execute the step of writing the first encrypted value as a query value into the linked list structure associated with the query key.

[0123] In some embodiments, the storage module 920 is further configured to determine whether the number of query values ​​included in the first database exceeds a preset number threshold; if the preset number threshold is exceeded, the linked list structure associated with the query key is adjusted to a tree structure, and the first encrypted value is written as a query value into the tree structure associated with the query key.

[0124] In some embodiments, the storage module 920 is further configured to associate and store the first encrypted value and the data identifier of the first type of data in an index set.

[0125] The data storage device provided in this application embodiment encrypts the query field to obtain the query key and quickly retrieves the corresponding query value in the first database. This query value can then be used to retrieve the corresponding first-type data in the second database as the query result corresponding to the query field. By storing multiple first-type data and their corresponding query keys, as well as the mapping relationship between query values ​​and query keys, in different databases, compared to related technologies that require decryption of the first-type data in the database to query the data, this device transforms the data query process from utilizing the mapping relationship between the query field and the first-type data into a mapping relationship between the data and its encrypted query key and query value. This effectively ensures privacy during the query process and improves data query efficiency because it eliminates the need to decrypt all data in the database.

[0126] Based on the same inventive concept, this application also provides a computer device, which can be a server or a terminal device. The computer device includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the aforementioned model evaluation method or model training method. This enables various functions, such as performing the following steps:

[0127] Encrypt the query fields to obtain the query key;

[0128] A query is performed in the first database based on the query key to obtain the query value corresponding to the query key;

[0129] Based on the query value, query the first type of data corresponding to the query value in the second database to obtain the query result corresponding to the query field. The second database is used to store multiple first type of data, and the first database is used to store the query value and query key corresponding to each first type of data.

[0130] Alternatively, perform the following steps:

[0131] Split the first type of data in the second database to obtain the first type of fields;

[0132] Encrypt the first type of data and the first type of field to obtain a first encrypted value of the first type of data and a second encrypted value of the first type of field;

[0133] The first encrypted value is used as the query value and the second encrypted value is used as the query key, and they are stored together in the first database.

[0134] The method provided in this application stores multiple types of first-type data and their query values ​​and query keys separately in a first database and a second database. This isolates the original data of the first-type data from the parameters used for querying, such as query keys and query values. Due to this isolation, the first-type data cannot be queried or viewed without prior knowledge of the preset encryption rules and the correspondence between the first and second databases, thus ensuring the security of the first-type data. When it is necessary to query the first-type data, the query field is encrypted using preset encryption rules to obtain the query key, and the corresponding query value is quickly retrieved in the first database. Then, the corresponding first-type data can be retrieved in the second database using this query value as the query result corresponding to the query field. By using different databases to store multiple types of first-type data and their corresponding query keys, as well as the mapping relationship between query values ​​and query keys, the data querying method, which previously relied on the mapping relationship between fields and data, is now transformed into a mapping relationship between data and its encrypted query keys and query values. This effectively ensures privacy during the query process and improves data query efficiency because it eliminates the need to decrypt all data in the database.

[0135] In one embodiment, taking a terminal device as an example, its internal structure diagram can be as follows: Figure 10 As shown, the computer device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a data query method or a data storage method. The display unit of the computer device is used to form a visually visible image. It can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0136] Those skilled in the art will understand that Figure 10 The structure shown is only a block diagram of a part of the structure related to the present application and does not constitute a limitation on the computer device on which the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0137] Based on the same inventive concept, embodiments of this application also provide a computer-readable storage medium, which may include: read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, etc.

[0138] Since the computer program stored in the computer-readable storage medium can execute any of the data query methods or data storage methods provided in the embodiments of this application, it can achieve the beneficial effects that any of the data query methods or data storage methods provided in the embodiments of this application can achieve, as detailed in the preceding embodiments, and will not be repeated here.

[0139] The solution adopted in this application stores multiple types of first-type data and their query values ​​and query keys separately in a first database and a second database. This isolates the original data of the first-type data from the parameters used for querying, such as query keys and query values. Due to this isolation, the first-type data cannot be queried or viewed without prior knowledge of the preset encryption rules and the correspondence between the first and second databases, thus ensuring the security of the first-type data. When it is necessary to query the first-type data, the query field is encrypted using preset encryption rules to obtain the query key, and the corresponding query value is quickly retrieved in the first database. Then, the corresponding first-type data can be retrieved in the second database using this query value as the query result corresponding to the query field. By using different databases to store multiple types of first-type data and their corresponding query keys, as well as the mapping relationship between query values ​​and query keys, the data querying method, which relies on the mapping relationship between fields and data, is transformed into a data querying method that relies on the mapping relationship between data and its fields after encryption, which relies on the mapping relationship between query keys and query values. This effectively ensures privacy during the query process and improves data query efficiency because it does not require decryption of all data in the database.

[0140] Based on the same inventive concept, embodiments of this application also provide a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in the various optional implementations of the above embodiments.

[0141] It should be noted that the object data (including but not limited to user device information, user personal information, etc.) and dialogue data involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of relevant countries and regions. Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods.

[0142] Any reference to memory, database, or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0143] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0144] In the above embodiments of the data query device or data storage device, computer-readable storage medium, computer equipment, and computer program product, the descriptions of each embodiment have different focuses. Parts not described in detail in a particular embodiment can be referred to in the relevant descriptions of other embodiments. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process and beneficial effects of the configuration item query device or configuration item update device, computer-readable storage medium, computer program product, computer equipment, and their corresponding units described above can be referred to the descriptions of the configuration item query method or configuration item update method in the above embodiments, and will not be repeated here.

[0145] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0146] The foregoing has provided a detailed description of a data query and storage method, apparatus, computer device, and storage medium provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A data query method, characterized in that: include: Encrypt the query field according to the preset encryption rule to obtain the query key; Searching the first database according to the query key to obtain a query value corresponding to the query key; According to the query value, the first type data corresponding to the query value is queried in the second database to obtain the query result corresponding to the query field, the second database is used to store multiple first type data, and the first database is used to store the query value and query key corresponding to each first type data.

2. The method according to claim 1, characterized in that The querying in the first database according to the query key to obtain a query value corresponding to the query key includes: Determine a third database from a plurality of first databases according to a field type of the query field; A query is performed in the third database according to the query key to obtain a query value corresponding to the query key.

3. The method according to claim 1, characterized in that The querying the first type of data corresponding to the query value in the second database according to the query value to obtain a query result corresponding to the query field includes: Determining a data identifier associated with the query value; The first type of data corresponding to the query value is queried in the second database according to the data identifier to obtain a query result corresponding to the query field.

4. The method according to claim 3, characterized in that The query field is obtained by parsing the query statement; The querying the first type of data corresponding to the query value in the second database according to the data identifier to obtain a query result corresponding to the query field includes: Replacing the query field in the query statement with the data identifier to obtain an updated query statement; A query is performed on the second database according to the updated query statement, and the first type of data obtained by the query is used as the query result.

5. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: Querying the cache filter according to the query key; In the case where the query key exists in the cache filter, executing the step of querying in the first database according to the query key to obtain a query value corresponding to the query key; If the query key does not exist in the cache filter, no query data is returned.

6. A data storage method, characterized in that: include: Splitting the first type of data in the second database to obtain the first type of fields; Encrypt the first type of data and the first type of field according to a preset encryption rule to obtain a first encrypted value of the first type of data and a second encrypted value of the first type of field; The first encrypted value is used as a query value and the second encrypted value is used as a query key, and the two are associated and stored in a first database.

7. The method according to claim 6, characterized in that The method further comprises: Determining whether the number of query values ​​included in the first database exceeds a preset number threshold; When the number exceeds a preset threshold, the linked list structure associated with the query key is adjusted to a tree structure, and the first encrypted value is written as a query value into the tree structure associated with the query key.

8. A data query device, characterized in that: include: An encryption module is used to encrypt the query field according to a preset encryption rule to obtain a query key; A query module, configured to query the first database according to the query key to obtain a query value corresponding to the query key; The query module is also used to query the first type of data corresponding to the query value in the second database according to the query value to obtain the query result corresponding to the query field, the second database is used to store multiple first type data, and the first database is used to store the query value and query key corresponding to each first type data.

9. A data storage device, characterized in that: include: A splitting module, used for splitting the first type of data stored in the second database to obtain the first type of fields; an encryption module, used to encrypt the first type of data and the first type of field according to a preset encryption rule to obtain a first encrypted value of the first type of data and a second encrypted value of the first type of field; The storage module is used to use the first encrypted value as a query value and the second encrypted value as a query key, and store them in a first database in an associated manner.

10. A computer device, characterized in that: It comprises a processor and a memory, wherein the memory stores a plurality of instructions; the processor loads instructions from the memory to execute the steps of the data query method according to any one of claims 1 to 5 or the steps of the data storage method according to any one of claims 6 to 7.

Images