Safety protection method, device, system and equipment for data lake warehouse and storage medium

Abstract

A security protection method, apparatus, system and device for a data lake bin and a storage medium are applied to the technical field of data security, and the method applied to a code obfuscation agent layer arranged between an interaction end and a file system layer comprises the following steps: in response to an interaction request initiated by the interaction end, performing user authentication on the interaction end to obtain a user authentication result; if the user authentication result indicates that the authentication is passed, responding to a data operation initiated by the interaction end, and performing anomaly detection on the data operation to obtain an anomaly detection result; when the abnormal detection result is abnormal operation, carrying out secondary authentication on the interaction end to obtain a secondary authentication result; and if the secondary authentication result is that the authentication is passed, executing data operation. According to the method, passive defense is upgraded into active immunization through an intermediate agent and a dual authentication mode based on an abnormal detection result, so that specific logic of data encryption, authentication and the like can be effectively hidden, the system is more difficult to crack and intrude, and better real-time data protection is provided.

Description

Technical Field

[0001] This application relates to the field of network security technology, specifically to a method, apparatus, system, device, and storage medium for the security protection of a data lake warehouse. Background Technology

[0002] In the digital age, database security issues are becoming increasingly prominent, especially database breaches (data theft) and ransomware attacks (data corruption), which pose significant risks and losses to businesses and individuals. While existing database encryption and protection technologies can protect data from unauthorized access to some extent, they typically have numerous problems when facing breaches and ransomware attacks. These include easily exposed encryption logic, inadequate key management, lack of restrictions on high-privilege user operations, and an inability to effectively prevent data corruption. These problems provide attackers with opportunities to exploit vulnerabilities.

[0003] Existing database protection solutions mainly cover the following aspects: Firstly, regarding data encryption protection, it can be divided into encrypting and desensitizing database data at runtime and encrypting data files during storage to enhance data security and privacy. However, its security is limited to the database itself and lacks resistance to unauthorized access at the database call layer, database hijacking, and brute-force ransomware encryption of data files. Secondly, while malware and malicious traffic can be detected and blocked to prevent malicious software and commands from damaging the database, malware and traffic detection relies on signature databases, resulting in a certain false positive and false negative rate. In particular, there is a lack of effective detection capabilities for new family signatures, bypass mechanisms, and zero-day vulnerabilities. Malware detection can only be used as one of the protection methods, not the primary one. Third, machine learning technology can be used to detect and block malicious database operations, and to promptly discover and prevent potential security threats. However, such methods generally rely on machine learning to identify malicious database operations and illegal operations on data files. The accuracy of the detection depends on the selection of samples and parameters. If the samples rely too much on the entire database data, real-time detection of the status of the entire data in a large data warehouse will consume too many resources and cannot guarantee good real-time performance.

[0004] Therefore, there is an urgent need for a data security protection method to effectively prevent data destruction. Summary of the Invention

[0005] At least one embodiment of this application provides a method, apparatus, system, device, and storage medium for the security protection of a data lake warehouse, which addresses the problem that the prior art cannot provide real-time and effective protection for data.

[0006] To solve the above-mentioned technical problems, this application is implemented as follows: In a first aspect, embodiments of this application provide a security protection method for a data lake warehouse, applied to a code obfuscation proxy layer set between the interactive end and the file system layer, including: In response to an interaction request initiated by the interactive terminal, user authentication is performed on the interactive terminal to obtain the user authentication result; If the user authentication result is determined to be successful, then respond to the data operation initiated by the interactive terminal, and perform anomaly detection on the data operation to obtain the anomaly detection result; If the anomaly detection result is an abnormal operation, a second authentication is performed on the interactive terminal to obtain a second authentication result. If the secondary authentication result is successful, then the data operation is performed.

[0007] Specifically, in the method described above, the step of performing anomaly detection on the data operation to obtain anomaly detection results includes: Based on preset content matching rules and statistical rules, high-risk operation detection is performed on the data operation to obtain a first detection result; Anomaly detection is performed on the data operation based on a pre-trained machine learning-based anomaly behavior detection model to obtain a second detection result; When both the first and second detection results are passed, the abnormal detection result is determined to be passed.

[0008] Specifically, in the method described above, the secondary authentication of the interactive terminal to obtain the secondary authentication result includes: Generate secure random numbers; The combination of the secure random number and the salt value is encrypted using a pre-determined public key to obtain encrypted data; The encrypted data is converted into verification data with a preset verification method and sent to the interactive terminal; The system receives decryption information sent by the interactive terminal, wherein the decryption information is obtained by the interactive terminal after decrypting the verification data according to the preset verification method. If the decryption information matches the secure random number and the salt value, then the secondary authentication result is determined to be successful.

[0009] Specifically, in the method described above, performing the data operation includes: Determine the data storage location of the target data corresponding to the data operation; If the data storage location is a file disk in the file system layer, then the target data is encrypted and decrypted according to a preset transparent data encryption strategy; If the data storage location is a data lake, then the data type and / or specific data bucket of the target data are subject to differentiated encryption and decryption.

[0010] Preferably, in the method described above, the differential encryption / decryption processing of the data type and / or data bucket index of the target data includes: Determine the corresponding encryption method based on the data type of the target data; The corresponding encryption key and encryption parameters are determined based on the data bucket index of the target data; The target data is encrypted and decrypted according to the encryption method, the encryption key, and the encryption parameters.

[0011] Furthermore, in the method described above, determining the corresponding encryption method based on the data type of the target data includes at least one of the following: If the data type of the target data is structured data, then column-level encryption is used; If the outermost data type of the target data is a compressed format in unstructured data, and the file size of the target data is less than or equal to a predetermined reference file size, then the encryption method is determined to be full encryption; otherwise, the encryption method is determined to be partial encryption, which only encrypts the beginning part of the compressed data block in the target data. The reference file size is calculated by statistically analyzing the encryption overhead of a preset number of file samples. If the data type of the target data is a media file in unstructured data and contains key data that needs to be protected, then the encryption method is determined to be partial encryption that adaptively encrypts the key data according to the specific type of the media file; otherwise, the encryption method is determined to be full encryption. If the data type of the target data is a text file in unstructured data, then the encryption method is determined to be full encryption.

[0012] Preferably, the method described above further includes: When the data operation is a data query or data export, the corresponding data is desensitized according to the preset desensitization configuration file.

[0013] Preferably, the method described above further includes: According to the system-level command, the code obfuscation proxy layer sets file protection permissions for files in the file system layer. The file protection permissions include: hiding permission, backup permission, and anti-modification permission. If a data operation modifying the file protection permissions is detected on the interactive terminal, the secondary authentication of the interactive terminal is triggered.

[0014] Preferably, the method described above further includes: Based on a preset obfuscation method, the preprocessing and / or compilation process in the code obfuscation proxy layer is obfuscated. The preset obfuscation method includes at least one of data obfuscation, character obfuscation, code obfuscation, and instruction obfuscation.

[0015] Specifically, in the method described above, the code obfuscation includes at least one of the following: By inserting spurious control flow, spurious condition judgments and / or spurious loops are performed; The linear execution flow can be divided into multiple parts by using jump statements and / or branch statements, and the execution order of the program can be changed by setting flags and loops; By setting inline functions, the compiled sub-function code is expanded in place to form a very large function block; By setting a generic function pointer, multiple functions can be called at runtime through the generic function pointer.

[0016] Secondly, embodiments of this application provide a control device applied to a code obfuscation proxy layer disposed between the interactive end and the file system layer, comprising: The user authentication module is used to respond to the interaction request initiated by the interaction terminal, perform user authentication on the interaction terminal, and obtain the user authentication result. An anomaly detection module is used to respond to the data operation initiated by the interactive terminal if the user authentication result is determined to be authentication passed, and to perform anomaly detection on the data operation to obtain an anomaly detection result. The secondary authentication module is used to perform secondary authentication on the interactive terminal when the anomaly detection result is an abnormal operation, and obtain the secondary authentication result. The execution module is used to perform the data operation if the secondary authentication result is successful.

[0017] Thirdly, embodiments of this application provide a data security system, including: a file system layer and a code obfuscation proxy layer, wherein the code obfuscation proxy layer connects the file system layer and the interactive terminal, and the code obfuscation proxy layer can implement the steps of the data lake warehouse security protection method applied to the code obfuscation proxy layer as described above.

[0018] Fourthly, embodiments of this application provide an electronic device, including: a processor, a memory, and a program stored in the memory and executable on the processor, wherein when the program is executed by the processor, it implements the steps of the security protection method for a data lake warehouse applied to a code obfuscation proxy layer as described above.

[0019] Fifthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the security protection method for a data lake warehouse applied to a code obfuscation proxy layer as described above.

[0020] In a sixth aspect, embodiments of this application provide a computer program product, including computer instructions that, when executed by a processor, implement the steps of the security protection method for a data lake warehouse applied to a code obfuscation proxy layer as described above.

[0021] Compared with the prior art, the data lake warehouse security protection method, device, system, equipment and storage medium provided in this application upgrades passive defense to active immunity by using an intermediate proxy and dual authentication based on anomaly detection results. It can effectively hide the specific logic of data encryption, authentication, etc., making the system more difficult to crack and invade, and providing better data encryption protection and user permission protection. Attached Figure Description

[0022] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 This is one of the flowcharts illustrating the security protection method for a data lake warehouse applied to the code obfuscation proxy layer in this embodiment; Figure 2 This is the second flowchart illustrating the security protection method for the data lake warehouse applied to the code obfuscation proxy layer in this embodiment; Figure 3 This is the third flowchart illustrating the security protection method for the data lake warehouse applied to the code obfuscation proxy layer in this embodiment; Figure 4 This is the fourth flowchart illustrating the security protection method for the data lake warehouse applied to the code obfuscation proxy layer in this embodiment; Figure 5 This is a schematic diagram illustrating the logic of encrypting compressed formats in unstructured data in this embodiment; Figure 6 This is a schematic diagram of the code execution logic based on false condition judgment in this embodiment; Figure 7 This is a schematic diagram of the code execution logic based on a spurious loop in this embodiment; Figure 8 This is a schematic diagram of the code execution logic based on jump statements in this embodiment; Figure 9This is a schematic diagram of the code execution logic based on branching statements in this embodiment; Figure 10 This is a schematic diagram of the code execution logic based on general function pointers in this embodiment; Figure 11 This is a schematic diagram of the control device applied to the code obfuscation proxy layer in this embodiment; Figure 12 This is a schematic diagram of the data security system in this embodiment; Figure 13 This is a schematic diagram of the electronic device in this embodiment. Detailed Implementation

[0023] Exemplary embodiments of the present application will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this application will be thorough and complete, and will fully convey the scope of the present application to those skilled in the art.

[0024] The terms “first,” “second,” etc., used in the specification and claims of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus. The terms “and / or” in the specification and claims indicate at least one of the connected objects.

[0025] Please refer to Figure 1 This application provides a data lake warehouse security protection method, applied to a code obfuscation proxy layer set between the interactive end and the file system layer, including: Step S101: Respond to the interaction request initiated by the interactive terminal, perform user authentication on the interactive terminal, and obtain the user authentication result; Step S102: If the user authentication result is determined to be authentication passed, then respond to the data operation initiated by the interactive terminal, and perform anomaly detection on the data operation to obtain the anomaly detection result. Step S103: If the anomaly detection result is an abnormal operation, perform secondary authentication on the interactive terminal to obtain a secondary authentication result. Step S104: If the secondary authentication result is successful, then perform the data operation.

[0026] The data lake warehouse protection method provided in this implementation is applied to a code obfuscation proxy layer set between the interactive end and the file system layer. By hiding the data lake warehouse call interface through an intermediate proxy, users and applications cannot directly access the data lake warehouse, which helps to improve data security. At the same time, its operation logic is unknown to the upper-layer application and the lower-layer data lake warehouse.

[0027] In its implementation, the code obfuscation proxy layer first responds to the interaction request sent by the client, performs routine user authentication on the client, and determines whether the client has the necessary permissions to use the code obfuscation proxy layer based on the authentication result. This ensures the user's legitimacy and security, thus intercepting unauthorized data operations at the initial stage of the interaction and preventing most common attacks (such as brute-force attacks and phishing attacks), thereby protecting data security. If the user authentication result is successful, meaning the client is legitimate, data operations can proceed. To prevent abnormal or illegal data operations through the client, anomaly detection is also performed on the responded data operations to intercept risks at the moment the operation is initiated. This reduces the probability of successful attacks and prevents accidental deletion or leakage of data due to misoperation or internal threats, thus preventing automated attacks (such as bot scripts stealing data in bulk). If an anomaly detection result indicates an abnormal operation, real-time secondary authentication is required on the interactive end to enforce verification of high-risk data operations. The data operation will only be executed after the secondary authentication is successful. This can prevent users' normal operations from being frequently interrupted and can effectively prevent the abuse of stolen user credentials, further reducing the success rate of high-risk operations. At the same time, the dual authentication mechanism can more effectively ensure the safe and stable operation of the system and prevent malicious behavior from causing adverse effects on the system.

[0028] In summary, the embodiments of this application upgrade passive defense to active immunity by using an intermediate proxy and dual authentication based on anomaly detection results. This effectively hides the specific logic of data encryption and authentication, making the system more difficult to crack and intrude, and providing better real-time data encryption protection and user permission protection.

[0029] See Figure 2 Specifically, in the method described above, the step of performing anomaly detection on the data operation to obtain anomaly detection results includes: Step S201: Perform high-risk operation detection on the data operation according to preset content matching rules and statistical rules to obtain the first detection result; Step S202: Perform abnormal operation detection on the data operation according to the pre-trained machine learning-based abnormal behavior detection model to obtain a second detection result; Step S203: When both the first detection result and the second detection result are passed, the abnormal detection result is determined to be passed.

[0030] In this embodiment, when performing anomaly detection on data operations, it is preferable to adopt a detection method that combines two detection methods to improve the accuracy and scope of anomaly detection.

[0031] Specifically, the first detection method determines whether a data operation is a high-risk operation by using preset content matching rules and statistical rules, and obtains a first detection result. The detection targets of this first detection method cover the following types of operations: 1) Traversing key information, such as multiple accesses to the database-level information table INFORMATION_SCHEMA, is very likely an attacker probing the table name, column information, permission information, etc. by traversing the table. 2) Querying the full data, such as querying all fields of data in a table without any restrictions or false conditions, is most likely a case of data hijacking. 3) Modifying and deleting large amounts of data, and modifying and deleting database tables and fields without any restrictions, is highly likely to be malicious data manipulation. 4) Directly executing SQL text files: Directly operating on SQL text files is a common method used by many attackers. At the same time, directly obtaining SQL text files from external sources and executing them in the database may introduce malicious code or execute unreviewed and dangerous operations. 5) Database backup and recovery: These operations involve large amounts of data and can easily lead to data leaks, damage, and the introduction of malicious data.

[0032] Specifically, the first detection method involves performing abnormal operation detection on the data operation based on a pre-trained machine learning-based abnormal behavior detection model to obtain a second detection result. In this method, a powerful detection model is first generated based on a sample library constructed from a large number of normal and malicious user behaviors to identify abnormal data operation behaviors. This machine learning-based detection method can continuously learn and adapt to new behavior patterns, thereby improving the accuracy and efficiency of abnormal behavior detection.

[0033] In one specific embodiment, the obtained detection model combines peak convolution and Long Short-Term Memory (LSTM) networks to learn and classify traffic features in both time and space. Peak convolution identifies anomalous traffic features, while LSTM preserves the sequential nature of the feature sequences, leveraging hierarchical spatiotemporal features to capture and locate abnormal behavior. The following is an example of the steps implemented in this detection model: First, traffic data is converted into two-dimensional grayscale images on a session-by-session basis through normalization, serving as input to the convolutional neural network. A selected convolutional kernel is applied to compute convolutions on the session data, highlighting short-term anomalies to make them easier to detect. Edge detection and sharpening convolutional kernels are used together in the computation. Examples of the horizontal and vertical directions of the edge detection kernel and the sharpening kernel on a 3x3 matrix are as follows: , , .

[0034] In addition to the original session data, corresponding features such as mean and variance are also collected as input to the convolution to obtain new feature data. A depthwise separable convolution algorithm is used for the input signal. First, a depthwise convolution is performed, with the convolution kernel being... The result of depthwise convolution for: Then, pointwise convolution is performed, with the convolution kernel being... The result of depthwise convolution Perform pointwise convolution to obtain the final output signal. for: LSTM replaces the hidden units in recurrent neural networks with memory-gated units, highlighting the temporal dependencies between data. The model performs computational functions by filtering and updating the convolutional output signal.

[0035] First, calculate the weights, where W is the weight and b is the bias. and The previous output and the current input are respectively used to calculate the result using a formula. Range of state values: Then, the candidate cell state node C of the LSTM is calculated: Finally, the output result is calculated using the output gate: in, and They represent the first The state values ​​of the depthwise convolution and pointwise convolution corresponding to each input; Indicates the state of candidate cells. Indicates the current cell state; Output status; This is the output result.

[0036] It should be noted that the above introduction to LSTM can be found in existing technical content, and will not be repeated here.

[0037] See Figure 3 Specifically, in the method described above, the secondary authentication of the interactive terminal to obtain the secondary authentication result includes: Step S301: Generate secure random numbers; Step S302: Encrypt the combination of the secure random number and the salt value according to the predetermined public key to obtain encrypted data; Step S303: Convert the encrypted data into verification data with a preset verification method and send it to the interactive terminal; Step S304: Receive decryption information sent by the interactive terminal, wherein the decryption information is obtained by the interactive terminal after decrypting the verification data according to the preset verification method; Step S305: If the decryption information matches the security random number and the salt value, then the secondary authentication result is determined to be successful.

[0038] In this embodiment, the steps of secondary authentication are illustrated. During secondary authentication, the code obfuscation proxy layer first generates a secure random number and its corresponding salt value. Then, it encrypts this combination of the secure random number and salt value using a pre-determined public key to obtain encrypted data, ensuring the confidentiality of the secure random number and salt value. This encrypted data is then converted into verification data corresponding to a preset verification method and sent to the interactive terminal. The preset verification method includes, but is not limited to, QR codes and SMS verification codes. The method of sending the data to the interactive terminal varies depending on the verification method; for example, for a QR code, a QR code verification box can be directly displayed. The interactive terminal, upon receiving the verification data, decrypts it to obtain decrypted information and sends it to the code obfuscation proxy layer for verification. The decryption method used by the interactive terminal corresponds to the preset verification method; for example, if the verification data is a QR code, the interactive terminal decrypts it by scanning the code. The code obfuscation proxy layer matches the received decrypted information with the previously generated secure random number and salt value. If a match is found, the secondary authentication result is considered successful; otherwise, the authentication is considered unsuccessful. Because the dual authentication uses secure random numbers and pre-configured asymmetric public key encryption, the verification cannot be spoofed unless the code obfuscation proxy layer's encryption logic is cracked and function-level listening privileges are obtained. This dual authentication mechanism more effectively ensures the secure and stable operation of the system, preventing malicious behavior from adversely affecting it.

[0039] Specifically, in the method described above, performing the data operation includes: Determine the data storage location of the target data corresponding to the data operation; If the data storage location is a file disk in the file system layer, then the target data is encrypted and decrypted according to a preset transparent data encryption strategy; If the data storage location is a data lake, then the data type and / or specific data bucket of the target data are subject to differentiated encryption and decryption.

[0040] In this implementation, when performing data operations, the data storage location of the target data corresponding to the data operation is first determined, and different encryption and decryption strategies are applied to the target data based on different data storage locations.

[0041] If the data storage location is a file disk in the file system layer, encryption and decryption are performed based on a transparent data encryption strategy. The transparent data encryption strategy involves embedding encryption and decryption logic through input / output hook processes when the database storage engine performs read and write operations on the file disk in the file system layer. Encryption is completed before writing data, and decryption is performed after data is read and before being returned to the storage engine process. The specific encryption logic and keys are implemented by the code obfuscation proxy layer, which can provide good concealment.

[0042] In a preferred embodiment, to provide fast encryption and decryption operations, a custom variant of the lightweight symmetric block cipher (xXTEA) is used, with a verification operation added to the xXTEA encryption logic. This verification operation yields an encrypted block and a verification block, enhancing both concealment and verification functionality. The length of the encrypted block is calculated from the overall file length, and the verification value is calculated on the encrypted block to obtain the verification block.

[0043] If the data storage location is a data lake, then differentiated encryption and decryption are performed on the data type and / or specific data bucket of the target data based on a differentiated encryption strategy. Since the data lake uses a partitioned and bucketed structure for data storage, the data type and Extensible Hash are selected as calculation parameters for the data index value during differentiated encryption and decryption. After calculating the index value using these parameters, the data is then added to the data lake based on the index. Extensible Hash is a dynamic hashing method that uses a combination of directories and buckets to achieve data storage and supports dynamic expansion of the hash table. This partitioned and bucketed strategy establishes a one-to-one correspondence between the index and the bucket. Differential encryption processing uses this as a basis, providing different encryption processes with the bucket as the smallest unit. By applying different encryption processes to data in different partitions or even different buckets, data security can be further guaranteed. For ease of understanding by those skilled in the art, specific embodiments of the differentiated encryption and decryption steps are provided below.

[0044] See Figure 4 Preferably, in the method described above, the differential encryption / decryption processing of the data type and / or data bucket index of the target data includes: Step S401: Determine the corresponding encryption method based on the data type of the target data; Step S402: Determine the corresponding encryption key and encryption parameters based on the data bucket index of the target data; Step S403: Encrypt and decrypt the target data according to the encryption method, the encryption key, and the encryption parameters.

[0045] In this embodiment, the corresponding encryption method is first determined based on the data type of the target data. This method of encrypting only the important data portions can significantly reduce the amount of data that needs to be encrypted while ensuring security, thus reducing the impact of encryption and decryption on system performance. This encryption method includes, but is not limited to, full encryption and partial encryption. Then, based on the data bucket index of the target data, the encryption key and encryption parameters corresponding to the target data can be determined. This ensures that the encryption parameters and encryption keys are not completely identical between different partitions and different data buckets. Therefore, even if the encryption method is obtained through the corresponding data within one data bucket, it is impossible to decrypt the data in other data buckets. This further guarantees the security of data within other data buckets, further increasing the difficulty of cracking and reducing the possibility of being compromised. The data bucket index is the data index value obtained above based on the scalable hash.

[0046] To facilitate understanding by those skilled in the art, examples of encryption methods corresponding to various data types are provided below.

[0047] Furthermore, in the method described above, determining the corresponding encryption method based on the data type of the target data includes at least one of the following: If the data type of the target data is structured data, then column-level encryption is used; column-level encryption can ensure data security while minimizing the impact on data analysis.

[0048] If the outermost data type of the target data is a compressed format in unstructured data, and the file size of the target data is less than or equal to a predetermined reference file size, then the encryption method is determined to be full encryption; otherwise, the encryption method is determined to be partial encryption, which only encrypts the beginning part of the compressed data block in the target data. The reference file size is calculated by statistically analyzing the encryption overhead of a predetermined number of file samples. In one embodiment, Word documents, Excel spreadsheets, compressed files, etc., have an outermost compressed format in unstructured data. Since the entire compressed file is composed of compressed data and file information from smaller internal files, full encryption is selected for smaller files; for larger files, compressed data blocks are quickly located through structure parsing, and the beginning part of the compressed data block (i.e., the entity file header and the corresponding first compressed block) is encrypted. The encryption logic is as follows: Figure 5 As shown. The reference file size used for file size differentiation is preferably obtained by calculating the encryption overhead by selecting a preset number of file samples. This partial encryption method can effectively ensure file security, and partial encryption of large files can greatly reduce the encryption and decryption overhead.

[0049] If the target data is a media file in unstructured data and contains critical data that needs protection, then the encryption method is determined to be partial encryption, which adaptively encrypts the critical data according to the specific type of the media file; otherwise, the encryption method is determined to be full encryption. In one embodiment, for video files, keyframes are encrypted; for audio files, frame header data is encrypted; for Joint Photographic Experts Group (JPG) images, Distributed Hash Table (DHT) encryption is used; and for Portable Network Graphics (PNG) images, the beginning of the Image Data IDAT block is encrypted. For unencrypted, untransformed raw data such as Bitmap (BMP) images, which do not contain critical data that needs protection, full encryption is still used.

[0050] If the data type of the target data is a text file in unstructured data, then the encryption method is determined to be full encryption. In one embodiment, for plain text files such as plain text files (TXT), log files (LOG), extensible markup language (XML), and JavaScript object notation (JSON), these files are relatively small in size, and the loss of some data may not have a significant impact on the remaining parts; therefore, full encryption is performed for protection.

[0051] Preferably, the method described above further includes: When the data operation is a data query or data export, the corresponding data is desensitized according to the preset desensitization configuration file.

[0052] In this embodiment, when data operations require data output, such as data querying or data export, the corresponding data is anonymized according to a preset anonymization configuration file. Specifically, the anonymization process includes, but is not limited to, automatically and accurately identifying key sensitive data such as ID card numbers, bank card numbers, mobile phone numbers, and addresses based on field names and attributes and using pre-defined matching rules. Subsequently, a preset anonymization algorithm is used to anonymize this data according to its type, preventing sensitive data from being displayed and transmitted in plaintext, thereby avoiding the leakage of sensitive data. Furthermore, even if the data is leaked, it cannot be used, ensuring data security in both storage and use.

[0053] Preferably, the method described above further includes: According to the system-level command, the code obfuscation proxy layer sets file protection permissions for files in the file system layer. The file protection permissions include: hiding permission, backup permission, and anti-modification permission. If a data operation modifying the file protection permissions is detected on the interactive terminal, the secondary authentication of the interactive terminal is triggered.

[0054] In this implementation, the code obfuscation proxy layer also provides file system-level protection for data files. First, it sets file protection permissions for files in the file system layer based on system-level commands. These permissions include, but are not limited to, hiding permissions, backup permissions, and anti-modification permissions. Hiding permissions conceal the file address to prevent it from being found; anti-modification permissions ensure that even the root user cannot delete or modify the file; and backup permissions ensure that even if a file is deleted, the deleted content is preserved for later recovery. Furthermore, by modifying the program logic of the executable file corresponding to the system-level commands, special processing is added to the data lake warehouse storage area. If a data operation modifying file protection permissions is detected on the interactive end—that is, a non-proxy layer user attempting to modify the hidden permissions of the data lake disk, its subdirectories, or files—a secondary authentication is triggered on the interactive end; otherwise, an alarm is issued and the operation is blocked. This achieves file-level security and effectively resists malicious attacks such as database deletion and ransomware encryption originating from the file level.

[0055] Preferably, the method described above further includes: Based on a preset obfuscation method, the preprocessing and / or compilation process in the code obfuscation proxy layer is obfuscated. The preset obfuscation method includes at least one of data obfuscation, character obfuscation, code obfuscation, and instruction obfuscation.

[0056] In this implementation, during the preprocessing and / or compilation process in the code obfuscation proxy layer, obfuscation is performed according to a preset obfuscation method. This obfuscation process hides various technical details, increases the difficulty of cracking the code by hiding specific logic, and ensures the concealment of subsequent encryption algorithms, keys, user authentication, malicious behavior detection, and other logic.

[0057] In one implementation, the pre-defined obfuscation method includes at least one of the following: Data obfuscation involves replacing function and variable names with random values, garbled text, or other meaningless content, and fully utilizing overloading techniques to use functions with the same name, thereby maximizing the disruption to program readability.

[0058] Character obfuscation involves using encryption to process constants and fixed strings in the code, and then decrypting them at runtime to provide concealment for the constants and fixed strings, increasing the difficulty of locating and cracking the program logic.

[0059] Code obfuscation is the process of altering the logic of code without affecting its execution result, making the code more difficult to understand.

[0060] Instruction obfuscation is the process of making assembly instructions more difficult to understand by changing them (replacing instructions with equivalent combinations of other instructions). Inline assembly adds _asm_-qualified assembly instructions to high-level language code and replaces the assembly instructions in the compiled executable program, such as swapping addition and subtraction instructions, and swapping logical operation instructions.

[0061] Specifically, in the method described above, the code obfuscation includes at least one of the following: By inserting spurious control flow, spurious condition judgments and / or spurious loops are performed; The linear execution flow can be divided into multiple parts by using jump statements and / or branch statements, and the execution order of the program can be changed by setting flags and loops; By setting inline functions, the compiled sub-function code is expanded in place to form a very large function block; By setting a generic function pointer, multiple functions can be called at runtime through the generic function pointer.

[0062] In one specific embodiment, the specific steps of code obfuscation are illustrated, which include four operations: First, add spurious control flow, that is, insert some redundant code logic into the program that does not affect the program's execution result, such as performing spurious conditional judgments and / or spurious loops. Figure 6 and Figure 7 As shown, examples of spurious conditional statements and spurious loops are provided respectively. a) A spurious conditional statement inserts a new conditional branch at the original selection branch, executes a series of spurious codes, and then executes the original logic; b) A spurious loop embeds a large spurious loop into the original flow before executing the normal logic. While adding spurious control flow makes the program spend more time executing this code, it also makes the control flow of the code more complex and difficult to understand.

[0063] Secondly, by using jump statements and / or branch statements to break down the linear execution flow into multiple parts, and by setting flags and loops to change the execution order of the program, linear complexity is achieved. This makes the execution logic of the entire code appear to be constantly switching between multiple branches, interfering with the understanding of the program. Figure 8 and Figure 9As shown, examples are given for the execution flow of jump statements based on jump statements (such as the goto statement) and the execution flow of branch statements based on branch statements (such as the wsitch function). It should be noted that in the original flow, each step jumps sequentially according to the order of arrangement.

[0064] Then, by setting inline functions, the call to the compiled sub-function is transformed into the in-place expansion of the sub-function code, resulting in a super-large function block of over a thousand lines. This leads to a significant increase in local variables, making data and objects more difficult to track, and consequently making the code more difficult to understand.

[0065] Finally, dynamic calling, which uses function pointers or function objects, makes function calls more dynamic, such as... Figure 10 As shown, by constructing a generic function pointer and using this function pointer for runtime calls to as many functions as possible, the calling logic becomes more complex and harder to trace.

[0066] By employing the above steps, the code's execution logic is altered, making it more difficult to understand. This hides the specific logic behind the code's operation, increasing the difficulty of cracking it and ensuring the concealment of encryption algorithms, keys, user authentication, and malicious behavior detection. This effectively protects the data lake warehouse.

[0067] The various methods of the embodiments of this application have been described above. Apparatus for implementing the above methods will now be provided.

[0068] See Figure 11 Preferably, another embodiment of this application provides a control device applied to a code obfuscation proxy layer disposed between the interactive end and the file system layer, comprising: User authentication module 1101 is used to respond to the interaction request initiated by the interaction terminal, perform user authentication on the interaction terminal, and obtain the user authentication result. An anomaly detection module 1102 is used to respond to the data operation initiated by the interactive terminal if the user authentication result is determined to be authentication passed, and to perform anomaly detection on the data operation to obtain an anomaly detection result. The secondary authentication module 1103 is used to perform secondary authentication on the interactive terminal when the anomaly detection result is an abnormal operation, and obtain a secondary authentication result. The execution module 1104 is used to execute the data operation if the secondary authentication result is successful.

[0069] Specifically, in the control device described above, the anomaly detection module includes: The first detection unit is used to perform high-risk operation detection on the data operation according to preset content matching rules and statistical rules, and obtain a first detection result; The second detection unit is used to perform abnormal operation detection on the data operation according to a pre-trained machine learning-based abnormal behavior detection model, and obtain a second detection result. The first processing unit is configured to determine that the abnormal detection result is a pass when both the first detection result and the second detection result are passable.

[0070] Specifically, in the control device described above, the secondary authentication module includes: Random number generation unit, used to generate secure random numbers; An encryption unit is used to encrypt the combination of the secure random number and the salt value according to a predetermined public key to obtain encrypted data; A conversion unit is used to convert the encrypted data into verification data with a preset verification method and send it to the interactive terminal; The receiving unit is used to receive decryption information sent by the interactive terminal, wherein the decryption information is obtained by the interactive terminal after decrypting the verification data according to the preset verification method; The verification unit is used to determine that the secondary authentication result is successful if the decryption information matches the secure random number and the salt value.

[0071] Specifically, in the control device described above, the execution module includes: A location determination unit is used to determine the data storage location of the target data corresponding to the data operation; The first encryption / decryption unit is used to encrypt / decrypt the target data according to a preset transparent data encryption strategy if the data storage location is a file disk in the file system layer. The second encryption / decryption unit is used to perform differentiated encryption / decryption on the data type and / or specific data bucket of the target data if the data storage location is a data lake.

[0072] Preferably, in the control device described above, the second encryption / decryption unit includes: The first determining subunit is used to determine the corresponding encryption method based on the data type of the target data; The second determining subunit is used to determine the corresponding encryption key and encryption parameters based on the data bucket index of the target data; An encryption / decryption subunit is used to encrypt and decrypt the target data according to the encryption method, the encryption key, and the encryption parameters.

[0073] Furthermore, in the control device described above, determining the corresponding encryption method based on the data type of the target data includes at least one of the following: If the data type of the target data is structured data, then column-level encryption is used; If the outermost data type of the target data is a compressed format in unstructured data, and the file size of the target data is less than or equal to a predetermined reference file size, then the encryption method is determined to be full encryption; otherwise, the encryption method is determined to be partial encryption, which only encrypts the beginning part of the compressed data block in the target data. The reference file size is calculated by statistically analyzing the encryption overhead of a preset number of file samples. If the data type of the target data is a media file in unstructured data and contains key data that needs to be protected, then the encryption method is determined to be partial encryption that adaptively encrypts the key data according to the specific type of the media file; otherwise, the encryption method is determined to be full encryption. If the data type of the target data is a text file in unstructured data, then the encryption method is determined to be full encryption.

[0074] Preferably, the control device described above further includes: The data anonymization module is used to anonymize the corresponding data according to a preset data anonymization configuration file when the data operation is data query or data export.

[0075] Preferably, the control device described above further includes: The permission setting module is used to set file protection permissions for files in the file system layer according to the code obfuscation proxy layer described in the system-level command. The file protection permissions include: hiding permission, backup permission, and anti-modification permission. The second processing module is used to trigger the secondary authentication of the interactive terminal if it detects that the interactive terminal has modified the file protection permissions.

[0076] Preferably, the control device described above further includes: The obfuscation module is used to obfuscate the preprocessing and / or compilation process in the code obfuscation proxy layer based on a preset obfuscation method, wherein the preset obfuscation method includes at least one of data obfuscation, character obfuscation, code obfuscation and instruction obfuscation.

[0077] Specifically, in the method described above, the obfuscation module includes at least one of the following: The first code obfuscation unit is used to perform false condition judgments and / or false loops by inserting false control flow; The second code obfuscation unit is used to split the linear execution flow into multiple parts through jump statements and / or branch statements, and to change the execution order of the program by setting flags and loops; The third code obfuscation unit is used to expand the compiled sub-function code in place to form a very large function block by setting inline functions; The fourth code obfuscation unit is used to enable multiple functions to be called at runtime through the general function pointer by setting a general function pointer.

[0078] It should be noted that the control device in this embodiment corresponds to the security protection method for the data lake warehouse applied to the code obfuscation proxy layer described above. The implementation methods in the above embodiments are all applicable to the embodiments of this device and can achieve the same technical effect. The control device provided in this application embodiment can implement all the method steps implemented in the above method embodiments and can achieve the same technical effect. Here, the parts that are the same as those in the method embodiments and the beneficial effects will not be described in detail.

[0079] See Figure 12 Preferably, another embodiment of this application provides a data security system, including: a file system layer 100 and a code obfuscation proxy layer 200. The code obfuscation proxy layer 200 connects the file system layer 100 and the interaction terminal 300. The code obfuscation proxy layer 200 can implement the steps of the data lake warehouse security protection method applied to the code obfuscation proxy layer as described above, and can achieve the above-mentioned technical effects. To avoid repetition, it will not be described again here.

[0080] See Figure 13 Preferably, another embodiment of this application provides an electronic device 1300, including: a processor 1301, a memory 1302, and a program stored in the memory 1302 and executable on the processor 1301. When the program is executed by the processor 1301, it implements the steps of the security protection method for the data lake warehouse applied to the code obfuscation proxy layer as described above, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0081] Preferably, another embodiment of this application provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the steps of the security protection method for a data lake warehouse applied to a code obfuscation proxy layer as described above, and achieves the same technical effect. To avoid repetition, it will not be described again here. The computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0082] Preferably, another embodiment of this application provides a computer program product, including computer instructions, which, when executed by a processor, implement the steps of the security protection method for a data lake warehouse applied to a code obfuscation proxy layer as described above, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0083] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0084] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0085] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A security protection method for a data lake warehouse, applied to a code obfuscation proxy layer set between the interactive end and the file system layer, characterized in that, include: In response to an interaction request initiated by the interactive terminal, user authentication is performed on the interactive terminal to obtain the user authentication result; If the user authentication result is determined to be successful, then respond to the data operation initiated by the interactive terminal, and perform anomaly detection on the data operation to obtain the anomaly detection result; If the anomaly detection result is an abnormal operation, a second authentication is performed on the interactive terminal to obtain a second authentication result. If the secondary authentication result is successful, then the data operation is performed.

2. The method according to claim 1, characterized in that, The anomaly detection process for the data operation, to obtain anomaly detection results, includes: Based on preset content matching rules and statistical rules, high-risk operation detection is performed on the data operation to obtain a first detection result; Anomaly detection is performed on the data operation based on a pre-trained machine learning-based anomaly behavior detection model to obtain a second detection result; When both the first and second detection results are passed, the abnormal detection result is determined to be passed.

3. The method according to claim 1, characterized in that, The secondary authentication process performed on the interactive terminal to obtain the secondary authentication result includes: Generate secure random numbers; The combination of the secure random number and the salt value is encrypted using a pre-determined public key to obtain encrypted data; The encrypted data is converted into verification data with a preset verification method and sent to the interactive terminal; The system receives decryption information sent by the interactive terminal, wherein the decryption information is obtained by the interactive terminal after decrypting the verification data according to the preset verification method. If the decryption information matches the secure random number and the salt value, then the secondary authentication result is determined to be successful.

4. The method according to claim 1, characterized in that, The execution of the data operation includes: Determine the data storage location of the target data corresponding to the data operation; If the data storage location is a file disk in the file system layer, then the target data is encrypted and decrypted according to a preset transparent data encryption strategy; If the data storage location is a data lake, then the data type and / or specific data bucket of the target data are subject to differentiated encryption and decryption.

5. The method according to claim 4, characterized in that, The differential encryption / decryption processing of the data type and / or data bucket index of the target data includes: Determine the corresponding encryption method based on the data type of the target data; The corresponding encryption key and encryption parameters are determined based on the data bucket index of the target data; The target data is encrypted and decrypted according to the encryption method, the encryption key, and the encryption parameters.

6. The method according to claim 5, characterized in that, Determining the corresponding encryption method based on the data type of the target data includes at least one of the following: If the data type of the target data is structured data, then column-level encryption is used; If the outermost data type of the target data is a compressed format in unstructured data, and the file size of the target data is less than or equal to a predetermined reference file size, then the encryption method is determined to be full encryption; otherwise, the encryption method is determined to be partial encryption, which only encrypts the beginning part of the compressed data block in the target data. The reference file size is calculated by statistically analyzing the encryption overhead of a preset number of file samples. If the data type of the target data is a media file in unstructured data and contains key data that needs to be protected, then the encryption method is determined to be partial encryption that adaptively encrypts the key data according to the specific type of the media file; otherwise, the encryption method is determined to be full encryption. If the data type of the target data is a text file in unstructured data, then the encryption method is determined to be full encryption.

7. The method according to claim 1, characterized in that, Also includes: When the data operation is a data query or data export, the corresponding data is desensitized according to the preset desensitization configuration file.

8. The method according to claim 1, characterized in that, Also includes: According to the system-level command, the code obfuscation proxy layer sets file protection permissions for files in the file system layer. The file protection permissions include: hiding permission, backup permission, and anti-modification permission. If a data operation modifying the file protection permissions is detected on the interactive terminal, the secondary authentication of the interactive terminal is triggered.

9. The method according to any one of claims 1 to 8, characterized in that, Also includes: Based on a preset obfuscation method, the preprocessing and / or compilation process in the code obfuscation proxy layer is obfuscated. The preset obfuscation method includes at least one of data obfuscation, character obfuscation, code obfuscation, and instruction obfuscation.

10. The method according to claim 9, characterized in that, The code obfuscation includes at least one of the following: By inserting spurious control flow, spurious condition judgments and / or spurious loops are performed; The linear execution flow can be divided into multiple parts by using jump statements and / or branch statements, and the execution order of the program can be changed by setting flags and loops; By setting inline functions, the compiled sub-function code is expanded in place to form a very large function block; By setting a generic function pointer, multiple functions can be called at runtime through the generic function pointer.

11. A control device applied to a code obfuscation proxy layer disposed between an interactive terminal and a file system layer, characterized in that, include: The user authentication module is used to respond to the interaction request initiated by the interaction terminal, perform user authentication on the interaction terminal, and obtain the user authentication result. An anomaly detection module is used to respond to the data operation initiated by the interactive terminal if the user authentication result is determined to be authentication passed, and to perform anomaly detection on the data operation to obtain an anomaly detection result. The secondary authentication module is used to perform secondary authentication on the interactive terminal when the anomaly detection result is an abnormal operation, and obtain the secondary authentication result. The execution module is used to perform the data operation if the secondary authentication result is successful.

12. A data security system, characterized in that, include: The system includes a file system layer and a code obfuscation proxy layer, wherein the code obfuscation proxy layer connects the file system layer and the interactive terminal, and the code obfuscation proxy layer can implement the steps of the data lake warehouse security protection method applied to the code obfuscation proxy layer as described in any one of claims 1 to 10.

13. An electronic device, characterized in that, include: A processor, a memory, and a program stored in the memory and executable on the processor, wherein the program, when executed by the processor, implements the steps of the security protection method for a data lake warehouse applied to a code obfuscation proxy layer as described in any one of claims 1 to 10.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the security protection method for a data lake warehouse applied to a code obfuscation proxy layer as described in any one of claims 1 to 10.

15. A computer program product, characterized in that, It includes computer instructions that, when executed by a processor, implement the steps of the security protection method for a data lake warehouse applied to a code obfuscation proxy layer as described in any one of claims 1 to 10.

Images