A data encryption method based on data warehouse

Abstract

The present invention discloses a data encryption method based on a data warehouse, which belongs to the field of data processing technology and includes the following steps: S1, generating parsing clues for a data warehouse; S2, generating parsing clue values ​​using the parsing clues according to the data type of the data to be encrypted; S3, obtaining a star model of the data to be encrypted and generating an identifier; S4, generating an encryption key using the parsing clue values ​​and the identifier of the data to be encrypted; S5, encrypting the data to be encrypted using the encryption key. The present invention proposes a data encryption method based on a data warehouse, which dynamically generates an encryption key according to the parsing clue values ​​and the identifier of the data to be encrypted. This method of dynamically generating a key increases the complexity and unpredictability of the key, thereby improving the security of the data.

Description

Technical Field

[0001] The present invention belongs to the technical field of data processing, and in particular relates to a data encryption method based on a data warehouse. Background Art

[0002] A data warehouse is a subject-oriented, integrated, non-updatable, and time-changing data collection that is used to better support the decision-making analysis and processing of an enterprise or organization. A data warehouse collects a large amount of internal or external data of an enterprise, which often contains a large amount of customer privacy data, such as name, gender, ID card, home address, zip code, mobile phone number, WeChat, family relationship, etc. In a data warehouse, data security plays an important role, because the leakage of privacy or sensitive data will cause serious damage to the property, reputation, personal safety, and legal interests of the data subject. Therefore, it is necessary to protect the data subject in the data warehouse. However, the existing data security protection methods are relatively low in security. Summary of the invention

[0003] In order to solve the above problems, the present invention proposes a data encryption method based on a data warehouse.

[0004] The technical solution of the present invention is: a data encryption method based on a data warehouse comprises the following steps:

[0005] S1, generate parsing clues for the data warehouse;

[0006] S2. Generate a parsing clue value using the parsing clue according to the data type of the data to be encrypted;

[0007] S3, obtain the star model of the data to be encrypted and generate an identifier;

[0008] S4, generating an encryption key using the parsing clue value and the identifier of the data to be encrypted;

[0009] S5. Encrypt the data to be encrypted using the encryption key.

[0010] Furthermore, S1 includes the following sub-steps:

[0011] S11. Calculate the amount of data annexation resources according to the update frequency of the data warehouse;

[0012] S12. Construct key analytical attributes for various data types in the data warehouse;

[0013] S13. Construct parsing clues based on the amount of data annexed resources and key parsing attributes of various data types.

[0014] Furthermore, in S11, the calculation formula for the data annexation resource amount N is:

[0015] Where D 1 represents the bandwidth of the data warehouse network configuration, D 2 represents the throughput of the data warehouse, f represents the data update frequency of the data warehouse, Indicates rounding up.

[0016] Furthermore, in S12, the data types include structured data, semi-structured data and unstructured data.

[0017] Furthermore, the key analytical attribute R of structured data all The calculation formula is:

[0018] ; In the formula, M k represents the code point of the kth character in the primary key corresponding to the structured data, O represents the code point of the foreign key corresponding to the structured data, K represents the total number of characters in the primary key corresponding to the structured data, HASH(·) represents the hash operation, and S represents the index of the structured data;

[0019] Key analytical properties of semi-structured data R half The calculation formula is:

[0020] ; Wherein, α represents the unique identifier of an object in the object exchange model of semi-structured data, and Y represents the value of an object in the object exchange model of semi-structured data;

[0021] Key analytical attributes of unstructured data R not The calculation formula is:

[0022] ; Wherein, F represents the query string corresponding to the seed URL for crawling unstructured data, and P represents the fragment identifier corresponding to the seed URL for crawling unstructured data.

[0023] The beneficial effect of the above further scheme is: in the present invention, in structured data, the primary key is the field or field combination that is uniquely used to identify a record in a database table, and each table can only have one primary key. The value of the primary key must be unique and cannot be a null value. The characters of the primary key are encoded, such as ASCII encoding, and each character corresponds to a unique numerical value. The database field data is converted into a fixed-length hash value using a Hash algorithm to perform key parsing attribute calculations.

[0024] Among semi-structured data, the object exchange model is a typical model of semi-structured data, which allows data items of the same type to have different sets of attributes, which is different from the requirement in traditional data models that all data of a certain type must have the same set of attributes. In the object exchange model, an object is represented by a triple consisting of a tag (or label), a type, and the value of the object. Objects in the object exchange model have unique identifiers and contain metadata that describes their structure and content. This allows data to be exchanged and stored without additional explanation or description.

[0025] Unstructured data refers to data that has no predefined data model, irregular data structure or incomplete data. Therefore, web crawler technology can be used to process unstructured data, such as Apache Nutch, which can be used to crawl unstructured data on the Internet. The seed URL is the starting point of the crawler, and Nutch will start crawling data from these seed URLs. When setting the query string of the seed URL, the user only needs to provide the query string and fragment identifier in the URL format.

[0026] Furthermore, in S13, the expression of the parsing clue Q is:

[0027] ; In the formula, N represents the amount of data annexed resources, i represents the data type, and R all Represents the key analytical attributes of structured data, R half Represents the key analytical attributes of semi-structured data, R not Represents the key parsing attributes of unstructured data, ALL represents structured data, HALF represents semi-structured data, NOT represents unstructured data, e represents an exponent, and c represents a constant.

[0028] Furthermore, S3 includes the following sub-steps:

[0029] S31, obtaining a star model of the data to be encrypted;

[0030] S32, split the dimension table of the star model into several dimension sub-tables;

[0031] S33. Calculate the identifier of the data to be encrypted according to the unique identifier of the fact table and the unique identifiers of several dimension sub-tables in the star model.

[0032] The beneficial effect of the above further scheme is that in the present invention, the star model is composed of a fact table and multiple dimension tables, the fact table is used to store the measurement data in the business process; the dimension table can be further divided into multiple sub-tables for describing the measurement data in the fact table to reduce data redundancy. The identifier calculation is completed using the specific identifiers of the fact table and the dimension table.

[0033] Furthermore, in S33, the expression of the identifier X of the data to be encrypted is:

[0034] ; where y represents the numerical index of the unique identifier of the fact table, and x h The numeric index representing the unique identifier of the h-th dimension subtable, x * It represents the mean of the unique identifiers of all dimension sub-tables. max(·) represents the maximum value function and e represents the exponent.

[0035] An index is a data structure used to quickly find and locate data. A numeric index is a numeric value.

[0036] The beneficial effects of the present invention are as follows: the present invention proposes a data encryption method based on a data warehouse, which dynamically generates an encryption key according to the parsing clue value and identifier of the data to be encrypted. This dynamic key generation method increases the complexity and unpredictability of the key, thereby improving the security of the data; in addition, the identifier is generated through a star model, and is combined with the parsing clue value to generate an encryption key. This process helps to ensure the consistency of the data before and after encryption, and prevent the data from being tampered with or damaged during the encryption process. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 The flowchart of the data encryption method based on data warehouse. DETAILED DESCRIPTION

[0038] The embodiments of the present invention will be further described below in conjunction with the accompanying drawings.

[0039] like Figure 1 As shown, the present invention provides a data encryption method based on a data warehouse, comprising the following steps:

[0040] S1, generate parsing clues for the data warehouse;

[0041] S2. Generate a parsing clue value using the parsing clue according to the data type of the data to be encrypted;

[0042] S3, obtain the star model of the data to be encrypted and generate an identifier;

[0043] S4, generating an encryption key using the parsing clue value and the identifier of the data to be encrypted;

[0044] S5. Encrypt the data to be encrypted using the encryption key.

[0045] In the embodiment of the present invention, the expression of the encryption key is (parsing clue value, identifier).

[0046] In this embodiment of the present invention, S1 includes the following sub-steps:

[0047] S11. Calculate the amount of data annexation resources according to the update frequency of the data warehouse;

[0048] S12. Construct key analytical attributes for various data types in the data warehouse;

[0049] S13. Construct parsing clues based on the amount of data annexed resources and key parsing attributes of various data types.

[0050] In the embodiment of the present invention, in S11, the calculation formula of the data swallowing resource amount N is:

[0051] Where D 1 represents the bandwidth of the data warehouse network configuration, D 2 represents the throughput of the data warehouse, f represents the data update frequency of the data warehouse, Indicates rounding up.

[0052] In the embodiment of the present invention, in S12, the data types include structured data, semi-structured data and unstructured data.

[0053] In the embodiment of the present invention, the key parsing attribute R of the structured data all The calculation formula is:

[0054] ; In the formula, M k represents the code point of the kth character in the primary key corresponding to the structured data, O represents the code point of the foreign key corresponding to the structured data, K represents the total number of characters in the primary key corresponding to the structured data, HASH(·) represents the hash operation, and S represents the index of the structured data;

[0055] Key analytical properties of semi-structured data R half The calculation formula is:

[0056] ; Wherein, α represents the unique identifier of an object in the object exchange model of semi-structured data, and Y represents the value of an object in the object exchange model of semi-structured data;

[0057] Key analytical attributes of unstructured data R not The calculation formula is:

[0058] ; Wherein, F represents the query string corresponding to the seed URL for crawling unstructured data, and P represents the fragment identifier corresponding to the seed URL for crawling unstructured data.

[0059] In the present invention, in structured data, the primary key is a field or field combination that is uniquely used to identify a record in a database table, and each table can only have one primary key. The value of the primary key must be unique and cannot be a null value. The characters of the primary key are encoded, such as ASCII encoding, and each character corresponds to a unique numerical value. The database field data is converted into a fixed-length hash value using a Hash algorithm to perform key parsing attribute calculations.

[0060] Among semi-structured data, the object exchange model is a typical model of semi-structured data, which allows data items of the same type to have different sets of attributes, which is different from the requirement in traditional data models that all data of a certain type must have the same set of attributes. In the object exchange model, an object is represented by a triple consisting of a tag (or label), a type, and the value of the object. Objects in the object exchange model have unique identifiers and contain metadata that describes their structure and content. This allows data to be exchanged and stored without additional explanation or description.

[0061] Unstructured data refers to data that has no predefined data model, irregular data structure or incomplete data. Therefore, web crawler technology can be used to process unstructured data, such as Apache Nutch, which can be used to crawl unstructured data on the Internet. The seed URL is the starting point of the crawler, and Nutch will start crawling data from these seed URLs. When setting the query string of the seed URL, the user only needs to provide the query string and fragment identifier in the URL format.

[0062] In the embodiment of the present invention, in S13, the expression for parsing clue Q is:

[0063] ; In the formula, N represents the amount of data annexed resources, i represents the data type, and R all Represents the key analytical attributes of structured data, R half Represents the key analytical attributes of semi-structured data, R not Represents the key parsing attributes of unstructured data, ALL represents structured data, HALF represents semi-structured data, NOT represents unstructured data, e represents an exponent, and c represents a constant.

[0064] In this embodiment of the present invention, S3 includes the following sub-steps:

[0065] S31, obtaining a star model of the data to be encrypted;

[0066] S32, split the dimension table of the star model into several dimension sub-tables;

[0067] S33. Calculate the identifier of the data to be encrypted according to the unique identifier of the fact table and the unique identifiers of several dimension sub-tables in the star model.

[0068] In the present invention, the star model consists of a fact table and multiple dimension tables. The fact table is used to store the measurement data in the business process; the dimension table can be further divided into multiple sub-tables to describe the measurement data in the fact table to reduce data redundancy. The identifier calculation is completed using the specific identifiers of the fact table and the dimension table.

[0069] In the embodiment of the present invention, in S33, the expression of the identifier X of the data to be encrypted is:

[0070] ; where y represents the numerical index of the unique identifier of the fact table, and x h The numeric index representing the unique identifier of the h-th dimension subtable, x * It represents the mean of the unique identifiers of all dimension sub-tables. max(·) represents the maximum value function and e represents the exponent.

[0071] An index is a data structure used to quickly find and locate data. A numeric index is a numeric value.

[0072] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and should be understood that the protection scope of the present invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific variations and combinations that do not deviate from the essence of the present invention based on the technical revelations disclosed by the present invention, and these variations and combinations are still within the protection scope of the present invention.

Claims

1. A data encryption method based on a data warehouse, characterized in that: The following steps are involved: S1, generate parsing clues for the data warehouse; S2. Generate a parsing clue value using the parsing clue according to the data type of the data to be encrypted; S3, obtain the star model of the data to be encrypted and generate an identifier; S4, generating an encryption key using the parsing clue value and the identifier of the data to be encrypted; S5. Encrypt the data to be encrypted using the encryption key; The S1 comprises the following sub-steps: S11. Calculate the amount of data annexation resources according to the update frequency of the data warehouse; S12. Construct key analytical attributes for various data types in the data warehouse; S13, constructing parsing clues based on the amount of data annexed resources and key parsing attributes of various data types; In S11, the calculation formula of the data annexation resource amount N is: ; In the formula, D1 represents the bandwidth of the data warehouse network configuration, D2 represents the throughput of the data warehouse, and f represents the data update frequency of the data warehouse. Indicates rounding up; In said S12, the data types include structured data, semi-structured data and unstructured data; The key analytical attribute R of the structured data all The calculation formula is: ; In the formula, M k represents the code point of the kth character in the primary key corresponding to the structured data, O represents the code point of the foreign key corresponding to the structured data, K represents the total number of characters in the primary key corresponding to the structured data, HASH(·) represents the hash operation, and S represents the index of the structured data; The key analytical attributes of the semi-structured data are R half The calculation formula is: ; Wherein, α represents the unique identifier of an object in the object exchange model of semi-structured data, and Y represents the value of an object in the object exchange model of semi-structured data; The key analytical attribute R of the unstructured data not The calculation formula is: ; Wherein, F represents the query string corresponding to the seed URL for crawling unstructured data, and P represents the fragment identifier corresponding to the seed URL for crawling unstructured data; In S13, the expression of the parsing clue Q is: ; In the formula, N represents the amount of data annexed resources, i represents the data type, and R all Represents the key analytical attributes of structured data, R half Represents the key analytical attributes of semi-structured data, R not Represents the key parsing attributes of unstructured data, ALL represents structured data, HALF represents semi-structured data, NOT represents unstructured data, e represents an exponent, and c represents a constant.

2. The data encryption method based on data warehouse according to claim 1 is characterized in that: The S3 comprises the following sub-steps: S31, obtaining a star model of the data to be encrypted; S32, split the dimension table of the star model into several dimension sub-tables; S33. Calculate the identifier of the data to be encrypted according to the unique identifier of the fact table and the unique identifiers of several dimension sub-tables in the star model.

3. The data encryption method based on data warehouse according to claim 2 is characterized in that: In S33, the expression of the identifier X of the data to be encrypted is: ; where y represents the numerical index of the unique identifier of the fact table, and x h The numeric index representing the unique identifier of the h-th dimension subtable, x * It represents the mean of the unique identifiers of all dimension sub-tables. max(·) represents the maximum value function and e represents the exponent.

Images