Database Engine Data Masking for Real-Time Transparent Protection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data protection methods in big-data environments face challenges such as high operational complexity, high costs, performance issues, and inability to provide real-time dynamic protection due to the use of external masking servers, which consume network bandwidth and resources, and require changes in application systems.
Innovation Solution
Integrate a data protection device within the big-data database execution engine to perform data masking and filtering operations, utilizing the distributed processing capability of the database engine without altering user request logic or data values, thereby eliminating the need for external masking servers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If an external masking server is used for data protection, then data protection function is achieved, but device complexity and operational difficulty increase
Solution Approach 1:
The patent merges the data protection function with the Hadoop ecosystem by integrating masking capabilities into MapReduce tasks. The masking server is combined with the Hadoop framework, allowing data protection to be performed as part of the normal data processing workflow without requiring separate external systems. This integration reduces operational complexity while maintaining data protection reliability.
Solution Approach 2:
The patent implements a universal data protection solution that works across multiple Hadoop components and data formats. The masking mechanism can be applied to various data types (text, numeric, binary) and integrates with different Hadoop operations (MapReduce, HDFS, Hive). This multi-functionality reduces the need for multiple specialized systems, thereby reducing overall device complexity.
2Reliability
If an external masking server is used for data protection, then data protection is achieved, but performance deteriorates due to network bandwidth consumption
Solution Approach 1:
The patent extracts the data masking operation from the network layer and implements it at the data processing layer within Hadoop. Instead of transmitting data through a masking server over the network, the masking is performed locally during MapReduce task execution. This extraction eliminates network bandwidth consumption while maintaining data protection functionality.
Solution Approach 2:
The Hadoop system performs data masking autonomously as part of its own processing mechanism. The MapReduce framework automatically applies masking rules to data during processing without requiring external intervention or network communication with a separate masking server. This self-service capability eliminates performance penalties associated with network transactions.
3Reliability
If an external masking server is used for data protection, then data protection is achieved, but deployment cost increases
Solution Approach 1:
The patent combines data protection functionality with existing Hadoop components, eliminating the need for separate masking server infrastructure. By merging masking capabilities into the Hadoop ecosystem, the solution leverages existing hardware and software investments rather than requiring additional dedicated systems, thereby reducing deployment costs.
Solution Approach 2:
The Hadoop system provides data protection services using its own processing resources rather than requiring external masking server infrastructure. The existing Hadoop cluster performs masking operations as part of normal data processing, eliminating the need for separate server hardware and reducing overall deployment costs.
Data Source
Figure 1~2
Figure 3~4
Figure 5~6
AI summary
Disclosed are a data protection method and device and a storage medium. The method includes that a database execution engine receives data request information sent by a client; a preset policy corresponding to an identity of the client carried in the data request information is acquired and the database execution engine performs a data protection operation for requested data according to the preset policy to obtain target data; and the database execution engine sends the target data to the client.