Data desensitization method and device, electronic equipment, storage medium and program product

Abstract

The invention provides a data desensitization method and device, electronic equipment, a storage medium and a program product, and relates to the technical field of data processing. Inputting an SQL query statement submitted by a user and query statement type information into the SQL analysis model, and outputting a query field corresponding to the SQL query statement; wherein the query statement type information is determined according to data lake cluster instance identity information input by a user; when it is detected that the query field needs to be subjected to desensitization processing, the query field needing to be subjected to desensitization processing and desensitization information corresponding to the query field are input into an SQL desensitization model, and a desensitization function corresponding to the query field is output; and according to the desensitization function, replacing the query field in the SQL query statement, and executing the replaced SQL query statement.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a data desensitization method, device, electronic device, storage medium and program product. Background Art

[0002] With the continuous development of digital technology, the protection of data privacy has become increasingly important, and many private data contain sensitive personal information, such as name, ID number, phone number, address, bank card number, etc. If these data are leaked or used improperly without processing, it will cause serious infringement of personal privacy.

[0003] In related technologies, privacy data desensitization processing relies on mature and well-established desensitization rules, and the rule configuration is complex. Therefore, how to desensitize data more effectively has become an urgent problem to be solved in the industry. Summary of the invention

[0004] The present invention provides a data desensitization method, device, electronic device, storage medium and program product to solve the problem of how to perform data desensitization more effectively in the prior art.

[0005] The present invention provides a data desensitization method, comprising the following steps: Inputting the SQL query statement and query statement type information submitted by the user into the SQL parsing model, and outputting the query field corresponding to the SQL query statement; wherein the query statement type information is determined according to the data lake cluster instance identity information input by the user; In the case where it is detected that the query field needs to be desensitized, the query field that needs to be desensitized and the desensitization information corresponding to the query field are input into the SQL desensitization model, and the desensitization function corresponding to the query field is output; According to the desensitizing function, the query field in the SQL query statement is replaced, and the replaced SQL query statement is executed.

[0006] According to a data desensitization method provided by the present invention, when it is detected that the query field needs to be desensitized, the query field to be desensitized and the desensitization information corresponding to the query field are input into an SQL desensitization model, and before the step of outputting the desensitization function corresponding to the query field, the method further includes: Encode the query field by using a hash algorithm to obtain a field encoding value of the query field; According to the field code value and the identity information of the user, matching the corresponding desensitized information in the user preset desensitization relationship table; When the desensitization type in the desensitization information is that desensitization is required, determining that the query field needs to be desensitized; Or, when the desensitization type in the desensitization information is that desensitization is not required, it is determined that the query field does not need to be desensitized.

[0007] According to a data desensitization method provided by the present invention, before the step of inputting the SQL query statement and query statement type information submitted by the user into the SQL parsing model and outputting the query field corresponding to the SQL query statement, the method further includes: Inputting the first language model prompt word into the general language model, and outputting a plurality of SQL query statement samples and query field labels corresponding to the SQL query statement samples; Taking each of the SQL query statement samples and the query field label corresponding to the SQL query statement sample as a first training sample, and obtaining a plurality of the first training samples; The preset model is trained based on the plurality of the first training samples to obtain the SQL parsing model.

[0008] According to a data desensitization method provided by the present invention, before the step of inputting the first language model prompt word into the general language model and outputting a plurality of SQL query statement samples and query field labels corresponding to the SQL query statement samples, the method further comprises: According to the low-rank adaptation technology, the general language model is fine-tuned through a large number of SQL query sentence samples carrying query field labels.

[0009] According to a data desensitization method provided by the present invention, before the step of inputting the query field to be desensitized and the desensitization information corresponding to the query field into an SQL desensitization model and outputting the desensitization function corresponding to the query field, the method further includes: Constructing a second language model prompt word according to the preset query field that needs to be desensitized, the preset desensitization level, and the query statement type information; The general language model is fine-tuned according to the second language model prompt word to obtain the SQL desensitization model.

[0010] According to a data desensitization method provided by the present invention, the desensitization information corresponding to the query field also includes: desensitization level information and query statement type information; the query field to be desensitized and the desensitization information corresponding to the query field are input into an SQL desensitization model, and a desensitization function corresponding to the query field is output, including: The query field that needs to be desensitized, as well as the desensitization level information and query statement type information corresponding to the query field are input into the SQL desensitization model to obtain a desensitization function corresponding to the SQL dialect type in the query statement type information.

[0011] The present invention also provides a data desensitization device, comprising the following modules: A query module, used to input the SQL query statement and query statement type information submitted by the user into the SQL parsing model, and output the query field corresponding to the SQL query statement; wherein the query statement type information is determined according to the data lake cluster instance identity information input by the user; A desensitization module is used for, when it is detected that the query field needs to be desensitized, inputting the query field that needs to be desensitized and the desensitization information corresponding to the query field into an SQL desensitization model, and outputting a desensitization function corresponding to the query field; A replacement module is used to replace the query field in the SQL query statement according to the desensitization function and execute the replaced SQL query statement.

[0012] According to a data desensitization device provided by the present invention, the device is also used for: Encode the query field by using a hash algorithm to obtain a field encoding value of the query field; According to the field code value and the identity information of the user, matching the corresponding desensitized information in the user preset desensitization relationship table; When the desensitization type in the desensitization information is that desensitization is required, determining that the query field needs to be desensitized; Or, when the desensitization type in the desensitization information is that desensitization is not required, it is determined that the query field does not need to be desensitized.

[0013] According to a data desensitization device provided by the present invention, the device is also used for: Inputting the first language model prompt word into the general language model, and outputting a plurality of SQL query statement samples and query field labels corresponding to the SQL query statement samples; Taking each of the SQL query statement samples and the query field label corresponding to the SQL query statement sample as a first training sample, and obtaining a plurality of the first training samples; The preset model is trained based on the plurality of the first training samples to obtain the SQL parsing model.

[0014] According to a data desensitization device provided by the present invention, the device is also used for: According to the low-rank adaptation technology, the general language model is fine-tuned through a large number of SQL query sentence samples carrying query field labels.

[0015] According to a data desensitization device provided by the present invention, the device is also used for: Constructing a second language model prompt word according to the preset query field that needs to be desensitized, the preset desensitization level, and the query statement type information; The general language model is fine-tuned according to the second language model prompt word to obtain the SQL desensitization model.

[0016] According to a data desensitization device provided by the present invention, the device is also used for: The query field that needs to be desensitized, as well as the desensitization level information and query statement type information corresponding to the query field are input into the SQL desensitization model to obtain a desensitization function corresponding to the SQL dialect type in the query statement type information.

[0017] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the data desensitization method described above is implemented.

[0018] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the data desensitization methods described above.

[0019] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the data desensitization method described in any one of the above is implemented.

[0020] The data desensitization method, device, electronic device, storage medium and program product provided by the present invention determine the type information of the query statement according to the cluster instance identity information, that is, the dialect type, the SQL query statement and the dialect type as input, input into the SQL parsing model, parse the SQL statement, identify the field information that needs to be queried, thereby realizing support for multiple SQL dialect statements, the query field that needs to be desensitized and the corresponding desensitization information, such as the desensitization level, are input into the SQL desensitization model, and the corresponding desensitization function is output. According to the desensitization function, the system automatically replaces the query field in the original SQL query statement and generates a new SQL query statement, ensuring that sensitive information is protected during the data query process, while the user can still perform necessary data analysis and processing. Through automated desensitization processing, the system improves data security and the efficiency and flexibility of data processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0022] Figure 1 It is a flow chart of the data desensitization method provided by the present invention; Figure 2 A schematic diagram of the processing structure provided by the present invention; Figure 3 The data desensitization processing flow chart provided by the present invention; Figure 4 A schematic diagram of the structure of the data desensitization device provided by the present invention; Figure 5 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] Figure 1 It is a flow chart of the data desensitization method provided by the present invention, such as Figure 1 As shown, the method includes the following: Step 110: Input the SQL query statement and query statement type information submitted by the user into the SQL parsing model, and output the query field corresponding to the SQL query statement; wherein the query statement type information is determined according to the data lake cluster instance identity information input by the user; In the present invention, the data lake cluster instance identity information is a unique identifier that identifies a specific data storage and computing environment. Based on this identity information, the system can determine the corresponding database dialect type, such as HIVE or MySQL, etc. This is because different data lake clusters may run different database systems, which have different SQL dialects.

[0025] The SQL parsing model is used to parse the input SQL query statement and identify the field information involved in the query. This model is usually based on complex algorithms or artificial intelligence technologies, such as big model technology, to understand and process SQL statements.

[0026] Once the SQL parsing model determines the field information in the SQL query statement, this information will be used as the basis for subsequent desensitization processing. For example, if the query involves sensitive information such as user identity information or phone number, these fields will be marked as requiring desensitization processing.

[0027] Step 120, when it is detected that the query field needs to be desensitized, input the query field that needs to be desensitized and the desensitization information corresponding to the query field into the SQL desensitization model, and output the desensitization function corresponding to the query field; In the present invention, the field information in the SQL query statement is evaluated to determine whether desensitization processing is required.

[0028] Once it is detected that a specific query field contains sensitive information and needs to be desensitized, these fields and their corresponding desensitization information will be input into the SQL desensitization model. The desensitization information may include the type of the field, desensitization rules, etc.

[0029] In the present invention, the SQL desensitization model is based on machine learning technology, especially large model technology, to understand and process complex desensitization logic models. It can generate corresponding desensitization functions according to the input query fields and desensitization information.

[0030] In the present invention, the desensitization function is a piece of code or query that can transform the original data to protect sensitive information. For example, for a field containing a personal name, the desensitization function may only display the first letter of the surname and the first letter of the given name (e.g., changing "Li X Ming" to "L.X").

[0031] In the present invention, the output of the SQL desensitization model is the desensitization function for each field that needs to be desensitized. These functions will be used to modify the original SQL query statement to ensure that the returned data is desensitized.

[0032] Step 130, according to the desensitization function, replace the query field in the SQL query statement and execute the replaced SQL query statement.

[0033] In the present invention, the desensitization function will be directly applied to the original SQL query statement. For example, if the desensitization function is MD5, then the original query field account.IDno will be replaced with this function.

[0034] In the present invention, by automatically scanning the SQL query statement, all fields that need to be desensitized are found and replaced with the corresponding desensitization functions. This process ensures that all sensitive data does not appear in plain text in the query results.

[0035] After the replacement operation is completed, the original SQL query statement is rewritten into a new query statement that contains desensitization functions. This new query statement will be used for subsequent database query operations.

[0036] The rewritten SQL query statement is sent to the data lake for execution. The data lake returns the query results, and the sensitive fields in these results have been processed by the desensitization function. For example, if the desensitization function is an encryption or masking operation, the returned results will show the encrypted or masked data.

[0037] The system returns the de-sensitized query results to the user, so that the user can safely use the data for analysis and processing without worrying about leaking sensitive information.

[0038] By executing anonymized query statements, enterprises ensure compliance with relevant data protection regulations and industry standards, such as GDPR, HIPAA, etc., during data processing and storage, while also protecting personal privacy and data security.

[0039] In the present invention, by determining the type information of the query statement according to the cluster instance identity information, that is, the dialect type, the SQL query statement and the dialect type are used as input, input into the SQL parsing model, the SQL statement is parsed, and the field information to be queried is identified, thereby realizing support for multiple SQL dialect statements, the query field to be desensitized and the corresponding desensitized information, such as the desensitization level, are input into the SQL desensitization model, and the corresponding desensitization function is output. According to the desensitization function, the system automatically replaces the query field in the original SQL query statement and generates a new SQL query statement, ensuring that sensitive information is protected during the data query process, while the user can still perform necessary data analysis and processing. Through automated desensitization processing, the system improves data security and the efficiency and flexibility of data processing.

[0040] Optionally, when it is detected that the query field needs to be desensitized, the query field that needs to be desensitized and the desensitization information corresponding to the query field are input into the SQL desensitization model, and before the step of outputting the desensitization function corresponding to the query field, the step further includes: Encode the query field by using a hash algorithm to obtain a field encoding value of the query field; According to the field code value and the identity information of the user, matching the corresponding desensitized information in the user preset desensitization relationship table; When the desensitization type in the desensitization information is that desensitization is required, determining that the query field needs to be desensitized; Or, when the desensitization type in the desensitization information is that desensitization is not required, it is determined that the query field does not need to be desensitized.

[0041] In the present invention, the desensitization verifier uses a hash algorithm (such as MD5, SHA-256, etc.) to encode the name or value of the query field. The hash algorithm can convert the input into a fixed-length output, which is called the field encoding value.

[0042] In the present invention, the user-preset desensitization relationship table stores the mapping relationship between field code values ​​and desensitization rules, as well as the desensitization configuration of each user or user group.

[0043] For example, as shown in Table 1 below: Table 1

[0044] The system uses the generated field code value together with the user's identity information as query conditions to search for matching desensitized information in the user's preset desensitized relationship table.

[0045] If the desensitization information found in the desensitization relationship table indicates that the desensitization type of the field is "demanding desensitization", the system determines that the query field needs to be desensitized. The system inputs the field and its desensitization information into the SQL desensitization model to generate the corresponding desensitization function.

[0046] If the desensitization information indicates that the desensitization type of the field is "no desensitization required", the system determines that the query field does not need to be desensitized. The system will directly execute the original SQL query statement without applying the desensitization function to these fields.

[0047] In the present invention, whether to perform desensitization processing on a specific field is dynamically determined based on the preset desensitization rules and the user's identity information, making the desensitization processing more accurate and efficient, while also improving the security and compliance of the system.

[0048] Optionally, before the step of inputting the SQL query statement and query statement type information submitted by the user into the SQL parsing model and outputting the query field corresponding to the SQL query statement, the method further includes: Inputting the first language model prompt word into the general language model, and outputting a plurality of SQL query statement samples and query field labels corresponding to the SQL query statement samples; Taking each of the SQL query statement samples and the query field label corresponding to the SQL query statement sample as a first training sample, and obtaining a plurality of the first training samples; The preset model is trained based on the plurality of the first training samples to obtain the SQL parsing model.

[0049] In the present invention, the first language model prompt word is a carefully designed instruction or question, which is used to guide the general language model to generate a specific SQL query statement sample.

[0050] For example, background: "You are a programmer who is proficient in HIVE and MYSQL syntax. Please generate question-answer pairs according to the following rules. The format is "Human: " + question +" Assistant: “+Answer+” Each question consists of a binary<SQL,TYPE> where SQL is a select query statement".

[0051] Requirements: 1. The SQL syntax and TYPE types correspond. 2. The given questions include both simple and complex SQL; For example: "select field 1, field 2 ... from table 1 join table 2" and TYPE is the dialect type of the SQL, for example: "MYSQL". The corresponding answer is the query content of the select statement, for example: "<table 1. field 1, table 2. field 2>".

[0052] In the present invention, the universal language model outputs multiple SQL query sample statements according to the input prompt words; at the same time, for each generated SQL query sample statement, the corresponding query field label needs to be output. These labels identify the location and type of each query field in the sample.

[0053] Each SQL query statement sample and its corresponding query field label are combined into a first training sample. In this way, each first training sample contains a SQL query statement and its corresponding field label, and these samples will be used to train the SQL parsing model. Repeating the above process, a large number of first training samples can be obtained, and these samples will form the basis of the training data set.

[0054] Based on the obtained multiple first training samples, a preset model (which may be a deep learning model or a machine learning model) is trained.

[0055] During the training process, the model learns how to identify and extract query field information from SQL query statements. After training, the preset model is transformed into a mature SQL parsing model that can understand the structure of SQL query statements and accurately identify and output query fields.

[0056] In the embodiment of the present invention, through the training and preparation process of the SQL parsing model, it is ensured that the model can accurately parse SQL query statements in practical applications and provide necessary field information for the subsequent data desensitization process.

[0057] Optionally, before the step of inputting the first language model prompt word into the universal language model and outputting a plurality of SQL query statement samples and query field labels corresponding to the SQL query statement samples, the method further comprises: According to the low-rank adaptation technology, the general language model is fine-tuned through a large number of SQL query sentence samples carrying query field labels.

[0058] In the present invention, low-rank adaptation is a technique for fine-tuning large pre-trained models by introducing low-rank matrices into the weight matrix of the model to adapt to new tasks or domains without adjusting all the parameters of the model. This method can reduce the number of parameters in the fine-tuning process, thereby reducing computational costs and improving efficiency.

[0059] During fine-tuning, a large number of SQL query sentence samples with query field labels are used to train the general language model. These samples include not only the SQL statements themselves, but also the corresponding query field labels, which indicate which fields in the statements need attention. By inputting these samples into the general language model, the model can learn how to identify and understand the query fields in SQL query statements. This process involves adjusting the weights of the model so that it can better adapt to the specific task, namely the identification of SQL query fields.

[0060] By inputting these samples into the general language model, the model can learn how to identify and understand the query fields in SQL query statements. This process involves adjusting the weights of the model so that it can better adapt to the specific task, namely the identification of SQL query fields.

[0061] In the present invention, the general language model is fine-tuned through low-rank adaptation technology, which can effectively improve the accuracy and adaptability of the model on specific tasks, and lay a solid foundation for the subsequent generation of SQL query statement samples and query field labels.

[0062] Optionally, before the step of inputting the query field to be desensitized and the desensitization information corresponding to the query field into the SQL desensitization model and outputting the desensitization function corresponding to the query field, the method further includes: Constructing a second language model prompt word according to the preset query field that needs to be desensitized, the preset desensitization level, and the query statement type information; The general language model is fine-tuned according to the second language model prompt word to obtain the SQL desensitization model.

[0063] In the present invention, the second language model prompt words are constructed according to the preset query field to be desensitized, the preset desensitization level and the query sentence type information. This step is to guide the general language model to generate sample data for a specific desensitization task.

[0064] Using the built second language model prompt words, fine-tune the general language model. This process involves using data specific to the desensitization task to adjust the parameters of the model so that it can better understand and handle desensitization-related queries.

[0065] After fine-tuning, the general language model is transformed into a specialized SQL desensitization model, which can output the corresponding desensitization function based on the input query fields and desensitization information.

[0066] Optionally, the second language model prompt word may specifically refer to: "You are a proficient {language} syntax, please use the SQL function {variable} is desensitized. "Requirements: 1. {rules}. 2. Directly output the desensitization function without any other content" Assistant: “ {function}( {variable})” Among them, language, variable, rules, and function are all variables, and fine-tuning test data is generated by replacement.

[0067] In an optional embodiment, based on the fine-tuning of the model, RAG is used to enhance the learning of SQL grammar of the big model, so as to understand the grammar of different SQL dialects, especially the use of different SQL functions. HIVE grammar documents, GBASE grammar documents and other documents are segmented and stored in a vector database to be provided to the big model for learning.

[0068] Optionally, the desensitization information corresponding to the query field also includes: desensitization level information and query statement type information; the query field to be desensitized and the desensitization information corresponding to the query field are input into an SQL desensitization model, and a desensitization function corresponding to the query field is output, including: The query field that needs to be desensitized, as well as the desensitization level information and query statement type information corresponding to the query field are input into the SQL desensitization model to obtain a desensitization function corresponding to the SQL dialect type in the query statement type information.

[0069] In the present invention, the query fields are fields that are determined to need to be desensitized. These fields contain sensitive information and need to be protected in the query results.

[0070] In the present invention, the masking level defines the strictness of the masking, for example, it may include partial masking, full masking or encryption, etc.

[0071] In the present invention, the query statement type information includes the SQL dialect type, indicating the database system (such as MySQL, PostgreSQL, Hive, etc.) to which the query statement is applicable, because different database systems may require different desensitization functions.

[0072] In an optional embodiment, four types of desensitization level information L1-L4 are predefined for different query field types, as shown in Table 2 below: Table 2

[0073] In the present invention, the SQL desensitization model identifies the query field of the input and understands the requirements of the desensitization level and SQL dialect. According to the input desensitization level information and SQL dialect type, the model generates a corresponding desensitization function. For example, if the dialect is Hive and the desensitization level is L2, the model may generate a desensitization function that uses the Hive SQL function.

[0074] In the present invention, the desensitization function output by the model is specific to the SQL dialect of the input, which means that for the same query field and desensitization level, different dialects may have different desensitization functions.

[0075] After obtaining the desensitization function, the generated desensitization function is applied to the original SQL query statement to replace the fields that need to be desensitized. In this way, when the query is executed, the sensitive data will be processed by the desensitization function, protecting the privacy and security of the data.

[0076] Figure 2 A schematic diagram of the processing structure provided by the present invention, such as Figure 2 As shown, it includes four parts: a SQL parsing processor, a SQL desensitization verifier, a SQL desensitization converter, and a SQL execution agent. The data gateway connects system users and the data lake.

[0077] The SQL parsing processor is responsible for the lexical and grammatical analysis of the input SQL query statement and identifying the field information in the SQL statement. It usually relies on the SQL parser based on large model technology and is trained using LoRA fine-tuning technology to improve parsing efficiency and accuracy.

[0078] The SQL desensitization verifier is used to determine whether desensitization is required when users query database table fields. Based on the results returned by the SQL parser and user information, the user field desensitization relationship table determines whether desensitization is required and the desensitization level.

[0079] The SQL desensitization converter is used to rewrite the query fields that need to be desensitized into SQL and generate desensitization functions. The SQL desensitization converter based on large model technology is trained using LoRA fine-tuning technology to train the model to rewrite SQL according to the desensitization level.

[0080] The SQL execution agent is used to route the desensitized SQL query statements to the cluster instance in the data lake for execution and return the results to the user. Based on the cluster identity information, the rewritten SQL is routed to the actual cluster for execution and the execution results are processed.

[0081] Figure 3 The data desensitization processing flow chart provided by the present invention is as follows: Figure 3 As shown, including: The user submits the pre-executed SQL and the identity information of the data lake cluster instance to the data gateway.

[0082] For example: a user queries the account information in the account table (account): SQL: "select IDno fromaccount"; ID: "BJHIVE"; The SQL parser determines the cluster type, GBASE or HIVE, based on the identity information of the data lake cluster instance. Then the SQL statement and dialect type are used as a tuple.<SQL,TYPE> The input is given to the SQL parsing model, and the SQL parsing model returns the library table field information that needs to be queried.

[0083] For example: SQL parsing large model input: <"select IDno from account", HIVE>; SQL parsing large model output:<account.IDno> .

[0084] The SQL desensitization verifier determines whether desensitization is required when user A queries the database table fields; If desensitization is required, the SQL desensitization converter constructs a prompt word based on the desensitization field, desensitization level, and dialect type and inputs it into the SQL desensitization big model. The big model returns the desensitization function FUNCTION (FIELD) under this dialect type.

[0085] For example: SQL desensitized large model input: "You are a SQL programmer who is proficient in HIVE syntax. Please use the SQL function to desensitize account.IDno." Require: 1. Use MD5 method to desensitize.

[0086] 2. Output the desensitization function directly without any other content; SQL desensitized large model output: MD5(account.IDno).

[0087] if<HIVE,MD5> If it is already in the cache, the desensitization function of the cache hit is directly used to return it.

[0088] If it is determined that desensitization operation is not required, the SQL execution agent routes the rewritten SQL to the actual cluster for execution based on the cluster identity information and returns the result to the user.

[0089] For example, the SQL execution agent replaces the original query field with the desensitizing function; select MD5(account.IDno)from account.

[0090] The SQL is sent to the actual physical cluster through a remote call for execution and waits for the execution result, and then the execution result is returned to the user.

[0091] In this invention, a data desensitization method based on large model technology is developed to adapt to the ever-changing data types and privacy protection requirements. A cross-platform SQL instruction conversion engine is implemented, and a conversion engine that can handle multiple SQL dialects is designed to achieve cross-platform data desensitization processing. It effectively protects personal privacy, ensures the compliance of enterprises when processing and storing data, and improves the efficiency and flexibility of data processing.

[0092] The data desensitization device provided by the present invention is described below. The data desensitization device described below and the data desensitization method described above can be referenced to each other.

[0093] Figure 4 The schematic diagram of the structure of the data desensitization device provided by the present invention is as follows: Figure 4 As shown, including: The query module 410 is used to input the SQL query statement and query statement type information submitted by the user into the SQL parsing model, and output the query field corresponding to the SQL query statement; wherein the query statement type information is determined according to the data lake cluster instance identity information input by the user; The desensitization module 420 is used to input the query field that needs to be desensitized and the desensitization information corresponding to the query field into the SQL desensitization model, and output the desensitization function corresponding to the query field when it is detected that the query field needs to be desensitized; The replacement module 430 is used to replace the query field in the SQL query statement according to the desensitization function and execute the replaced SQL query statement.

[0094] According to a data desensitization device provided by the present invention, the device is also used for: Encode the query field by using a hash algorithm to obtain a field encoding value of the query field; According to the field code value and the identity information of the user, matching the corresponding desensitized information in the user preset desensitization relationship table; When the desensitization type in the desensitization information is that desensitization is required, determining that the query field needs to be desensitized; Or, when the desensitization type in the desensitization information is that desensitization is not required, it is determined that the query field does not need to be desensitized.

[0095] According to a data desensitization device provided by the present invention, the device is also used for: Inputting the first language model prompt word into the general language model, and outputting a plurality of SQL query statement samples and query field labels corresponding to the SQL query statement samples; Taking each of the SQL query statement samples and the query field label corresponding to the SQL query statement sample as a first training sample, and obtaining a plurality of the first training samples; The preset model is trained based on the plurality of the first training samples to obtain the SQL parsing model.

[0096] According to a data desensitization device provided by the present invention, the device is also used for: According to the low-rank adaptation technology, the general language model is fine-tuned through a large number of SQL query sentence samples carrying query field labels.

[0097] According to a data desensitization device provided by the present invention, the device is also used for: Constructing a second language model prompt word according to the preset query field that needs to be desensitized, the preset desensitization level, and the query statement type information; The general language model is fine-tuned according to the second language model prompt word to obtain the SQL desensitization model.

[0098] According to a data desensitization device provided by the present invention, the device is also used for: The query field that needs to be desensitized, as well as the desensitization level information and query statement type information corresponding to the query field are input into the SQL desensitization model to obtain a desensitization function corresponding to the SQL dialect type in the query statement type information.

[0099] In an embodiment of the present invention, by determining the type information of the query statement according to the cluster instance identity information, that is, the dialect type, the SQL query statement and the dialect type are input as inputs, input into the SQL parsing model, the SQL statement is parsed, and the field information to be queried is identified, thereby achieving support for multiple SQL dialect statements, the query field to be desensitized and the corresponding desensitized information, such as the desensitization level, are input into the SQL desensitization model, and the corresponding desensitization function is output. According to the desensitization function, the system automatically replaces the query field in the original SQL query statement and generates a new SQL query statement, ensuring that sensitive information is protected during the data query process, while the user can still perform necessary data analysis and processing. Through automated desensitization processing, the system improves data security and the efficiency and flexibility of data processing.

[0100] Figure 5 is a schematic diagram of the structure of the electronic device provided by the present invention, such as Figure 5 As shown, the electronic device may include: a processor 510, a communication interface 520, a memory 530 and a communication bus 540, wherein the processor 510, the communication interface 520 and the memory 530 communicate with each other through the communication bus 540. The processor 510 may call the logic instructions in the memory 530 to execute the data desensitization method, which includes: inputting the SQL query statement and the query statement type information submitted by the user into the SQL parsing model, and outputting the query field corresponding to the SQL query statement; wherein the query statement type information is determined according to the data lake cluster instance identity information input by the user; In the case where it is detected that the query field needs to be desensitized, the query field that needs to be desensitized and the desensitization information corresponding to the query field are input into the SQL desensitization model, and the desensitization function corresponding to the query field is output; According to the desensitizing function, the query field in the SQL query statement is replaced, and the replaced SQL query statement is executed.

[0101] In addition, the logic instructions in the above-mentioned memory 530 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0102] On the other hand, the present invention further provides a computer program product, the computer program product includes a computer program, the computer program can be stored on a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the data desensitization method provided by the above methods, the method includes: inputting the SQL query statement and query statement type information submitted by the user into the SQL parsing model, and outputting the query field corresponding to the SQL query statement; wherein the query statement type information is determined according to the data lake cluster instance identity information input by the user; In the case where it is detected that the query field needs to be desensitized, the query field that needs to be desensitized and the desensitization information corresponding to the query field are input into the SQL desensitization model, and the desensitization function corresponding to the query field is output; According to the desensitizing function, the query field in the SQL query statement is replaced, and the replaced SQL query statement is executed.

[0103] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to execute the data desensitization method provided by the above methods, the method comprising: inputting a SQL query statement and query statement type information submitted by a user into a SQL parsing model, and outputting a query field corresponding to the SQL query statement; wherein the query statement type information is determined according to the data lake cluster instance identity information input by the user; In the case where it is detected that the query field needs to be desensitized, the query field that needs to be desensitized and the desensitization information corresponding to the query field are input into the SQL desensitization model, and the desensitization function corresponding to the query field is output; According to the desensitizing function, the query field in the SQL query statement is replaced, and the replaced SQL query statement is executed.

[0104] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0105] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0106] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A data desensitization method, characterized in that: include: Inputting the SQL query statement and query statement type information submitted by the user into the SQL parsing model, and outputting the query field corresponding to the SQL query statement; wherein the query statement type information is determined according to the data lake cluster instance identity information input by the user; In the case where it is detected that the query field needs to be desensitized, the query field that needs to be desensitized and the desensitization information corresponding to the query field are input into the SQL desensitization model, and the desensitization function corresponding to the query field is output; According to the desensitizing function, the query field in the SQL query statement is replaced, and the replaced SQL query statement is executed.

2. The data desensitization method according to claim 1, characterized in that: Before the step of inputting the query field to be desensitized and the desensitization information corresponding to the query field into the SQL desensitization model and outputting the desensitization function corresponding to the query field when it is detected that the query field needs to be desensitized, the method further includes: Encode the query field by using a hash algorithm to obtain a field encoding value of the query field; According to the field code value and the identity information of the user, matching the corresponding desensitized information in the user preset desensitization relationship table; When the desensitization type in the desensitization information is that desensitization is required, determining that the query field needs to be desensitized; Or, when the desensitization type in the desensitization information is that desensitization is not required, it is determined that the query field does not need to be desensitized.

3. The data desensitization method according to claim 1, characterized in that: Before the step of inputting the SQL query statement and query statement type information submitted by the user into the SQL parsing model and outputting the query field corresponding to the SQL query statement, the method further includes: Inputting the first language model prompt word into the general language model, and outputting a plurality of SQL query statement samples and query field labels corresponding to the SQL query statement samples; Taking each of the SQL query statement samples and the query field label corresponding to the SQL query statement sample as a first training sample, and obtaining a plurality of the first training samples; The preset model is trained based on the plurality of the first training samples to obtain the SQL parsing model.

4. The data desensitization method according to claim 3, characterized in that: Before the step of inputting the first language model prompt word into the universal language model and outputting a plurality of SQL query statement samples and query field labels corresponding to the SQL query statement samples, the method further includes: According to the low-rank adaptation technology, the general language model is fine-tuned through a large number of SQL query sentence samples carrying query field labels.

5. The data desensitization method according to claim 2, characterized in that: Before the step of inputting the query field to be desensitized and the desensitization information corresponding to the query field into the SQL desensitization model and outputting the desensitization function corresponding to the query field, the method further includes: Constructing a second language model prompt word according to the preset query field that needs to be desensitized, the preset desensitization level, and the query statement type information; The general language model is fine-tuned according to the second language model prompt word to obtain the SQL desensitization model.

6. The data desensitization method according to claim 5, characterized in that: The desensitization information corresponding to the query field also includes: desensitization level information and query statement type information; the query field that needs to be desensitized and the desensitization information corresponding to the query field are input into the SQL desensitization model, and the desensitization function corresponding to the query field is output, including: The query field that needs to be desensitized, as well as the desensitization level information and query statement type information corresponding to the query field are input into the SQL desensitization model to obtain a desensitization function corresponding to the SQL dialect type in the query statement type information.

7. A data desensitization device, characterized in that: include: A query module, used to input the SQL query statement and query statement type information submitted by the user into the SQL parsing model, and output the query field corresponding to the SQL query statement; wherein the query statement type information is determined according to the data lake cluster instance identity information input by the user; A desensitization module is used for inputting the query field to be desensitized and the desensitization information corresponding to the query field into the SQL desensitization model when it is detected that the query field needs to be desensitized, and outputting the desensitization function corresponding to the query field; A replacement module is used to replace the query field in the SQL query statement according to the desensitization function and execute the replaced SQL query statement.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the data desensitization method as described in any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the data desensitization method as described in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the data desensitization method as described in any one of claims 1 to 6 is implemented.

Images