A security protection method for big data

Abstract

This invention relates to the field of data security protection technology, and in particular to a method for big data security protection. The method includes the following steps: acquiring system operation data; mapping the storage locations of the system operation data to obtain a data location mapping table; performing real-time data encryption based on the data location mapping table to obtain an encrypted data storage table; acquiring system memory data; performing reference counting statistics on the system memory data based on the encrypted data storage table to obtain a data reference status table; performing residual detection analysis on the data reference status table to obtain a data residual marker set; performing multi-level cache clearing of residual data based on the data residual marker set to obtain a memory cleanup verification table; and performing defragmentation optimization on the memory cleanup verification table to obtain an optimized memory layout diagram. This invention effectively solves the weaknesses in dynamic data protection in existing technologies, significantly improving the security and reliability of big data systems.

Description

Technical Field

[0001] This invention relates to the field of data security protection technology, and in particular to a method for protecting big data security. Background Technology

[0002] In the field of data security, traditional encryption and access control mechanisms mainly focus on protecting static data, but research on dynamic protection of data during computation is relatively limited. Especially in distributed computing environments, the flow and computation of data between different nodes can easily lead to data leakage security risks.

[0003] Current big data security solutions suffer from a subtle but serious problem: when multiple computing tasks process the same dataset in parallel, the caching mechanism for intermediate results can easily lead to data remnants. For example, in a financial institution's risk control system, during real-time analysis of customer transaction data, some analysis tasks may be forcibly interrupted due to excessive system load, leaving some sensitive data in the memory of distributed nodes. This residual data not only consumes system resources but, more importantly, is easily accessed by malicious programs through memory scanning and other methods, resulting in data leaks.

[0004] Meanwhile, in big data analytics, data computing nodes typically use shared memory pools to improve computational efficiency. However, existing memory pool management mechanisms lack fine-grained data security marking and cleanup mechanisms. This leads to the confusion of sensitive data from different participants during intermediate computations in complex data analysis scenarios, such as federated learning tasks, and even data remaining in the memory pool after the task is completed, increasing the risk of data leakage. Summary of the Invention

[0005] Therefore, it is necessary for the present invention to provide a big data security protection method to solve at least one of the above-mentioned technical problems.

[0006] To achieve the above objectives, a big data security protection method includes the following steps:

[0007] Step S1: Obtain system operation data; map the storage location of the system operation data to obtain a data location mapping table;

[0008] Step S2: Perform real-time data encryption based on the data location mapping table to obtain an encrypted data storage table;

[0009] Step S3: Obtain system memory data; perform reference counting statistics on system memory data based on the encrypted data storage table to obtain a data reference status table; perform residual detection analysis on the data reference status table to obtain a data residual tag set; perform multi-level cache clearing of residual data based on the data residual tag set to obtain a memory cleanup verification table; perform defragmentation optimization on the memory cleanup verification table to obtain an optimized memory layout diagram;

[0010] Step S4: Based on the optimized memory layout diagram, construct a secure channel for the distributed nodes to obtain a node communication configuration table; perform task scheduling and allocation on the node communication configuration table to obtain a secure task allocation table; perform interrupted task data protection processing based on the secure task allocation table to obtain a task protection status table; perform node synchronization updates on the task protection status table to obtain a data consistency verification table.

[0011] Step S5: Perform real-time behavior monitoring of the distributed system based on the data consistency verification table to obtain a behavior anomaly detection table; adaptively adjust the protection strategy of the distributed system according to the behavior anomaly detection table to perform big data security protection.

[0012] This invention ensures the confidentiality and integrity of data during its flow and computation by mapping the storage location of system operation data and encrypting it in real time, preventing unauthorized access or tampering during transmission and storage. Secondly, based on reference counting statistics and residual detection analysis, it accurately identifies and removes residual data in memory, avoiding the risk of sensitive data leakage due to caching mechanisms. Simultaneously, it optimizes memory usage efficiency and reduces system resource consumption through a multi-level cache clearing strategy. By constructing secure communication channels and task scheduling and allocation mechanisms for distributed nodes, it ensures the integrity of interrupted task data and achieves data consistency verification between nodes, enhancing the stability and reliability of the distributed system in complex computing tasks. Finally, the real-time behavior monitoring and adaptive adjustment mechanism for protection strategies can dynamically respond to potential threats, promptly detect and prevent abnormal behavior, further strengthening security protection capabilities in big data environments. In summary, this invention addresses the entire process from data processing, storage, and transmission to monitoring, effectively solving the weaknesses in dynamic data protection in existing technologies and significantly improving the security and reliability of big data systems. Attached Figure Description

[0013] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description taken in conjunction with the accompanying drawings:

[0014] Figure 1 A flowchart illustrating the steps of a big data security protection method according to an embodiment is shown.

[0015] Figure 2 A detailed flowchart of step S15 of one embodiment is shown.

[0016] Figure 3 A detailed flowchart of step S25 of one embodiment is shown. Detailed Implementation

[0017] The technical method of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0018] Furthermore, the accompanying drawings are merely illustrative of the invention and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor methods and / or microcontroller methods.

[0019] It should be understood that although the terms "first," "second," etc., may be used herein to describe various units, these units should not be limited by these terms. These terms are used merely to distinguish one unit from another. For example, without departing from the scope of the exemplary embodiments, a first unit may be referred to as a second unit, and similarly, a second unit may be referred to as a first unit. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0020] To achieve the above objectives, please refer to Figures 1 to 3 This invention provides a method for protecting big data security, comprising the following steps:

[0021] Step S1: Obtain system operation data; map the storage location of the system operation data to obtain a data location mapping table;

[0022] Step S2: Perform real-time data encryption based on the data location mapping table to obtain an encrypted data storage table;

[0023] Step S3: Obtain system memory data; perform reference counting statistics on system memory data based on the encrypted data storage table to obtain a data reference status table; perform residual detection analysis on the data reference status table to obtain a data residual tag set; perform multi-level cache clearing of residual data based on the data residual tag set to obtain a memory cleanup verification table; perform defragmentation optimization on the memory cleanup verification table to obtain an optimized memory layout diagram;

[0024] Step S4: Based on the optimized memory layout diagram, construct a secure channel for the distributed nodes to obtain a node communication configuration table; perform task scheduling and allocation on the node communication configuration table to obtain a secure task allocation table; perform interrupted task data protection processing based on the secure task allocation table to obtain a task protection status table; perform node synchronization updates on the task protection status table to obtain a data consistency verification table.

[0025] Step S5: Perform real-time behavior monitoring of the distributed system based on the data consistency verification table to obtain a behavior anomaly detection table; adaptively adjust the protection strategy of the distributed system according to the behavior anomaly detection table to perform big data security protection.

[0026] In this embodiment, a risk control system of a financial institution is used as an example. The system's operational data includes customer transaction records, account balances, credit scores, and other information. The system's operational data is acquired using the Apache NiFi data flow management tool, and its data processing capabilities are used to map the data's storage location. Apache NiFi stores the data in different storage areas (such as HDFS or object storage) based on the data type and sensitivity, generating a data location mapping table. The Apache Ranger permission management tool, combined with the AES encryption algorithm, performs real-time encryption processing on the data based on the data location mapping table, generating an encrypted data storage table. System memory data is acquired through the Linux / proc file system, and the Prometheus monitoring system is used to perform reference counting statistics on the memory data. The Prometheus time-series database records the reference count and reference timestamp of each data block, generating a data reference status table. The Volatility memory analysis tool is used to perform residual detection analysis on the data reference status table, and the YARA rule engine is used to extract abnormal features, generating a data residual tag set. Based on the data residual tag set, the Ansible automation tool, combined with the Linux dd command and Python scripts, performs multi-level cache clearing of residual data. For low-priority data, the `dd` command is used to zero it directly. For medium-priority data, the `Rsync` tool is used for archiving and a 7-day backup is retained. For high-priority data, cleanup operations are dynamically triggered based on system memory usage. After cleanup, the Linux `md5sum` command is used for integrity verification, generating a memory cleanup verification table. The Ceph distributed storage system is used to defragment and optimize the memory cleanup verification table, generating an optimized memory layout diagram. Subsequently, OpenSSL is used to build a secure communication channel for distributed nodes, generating a node communication configuration table. The Apache Spark distributed computing framework is used to schedule and allocate tasks from the node communication configuration table, generating a secure task allocation table. For interrupted task data protection, Zookeeper is used to coordinate and ensure task state consistency, generating a task protection status table. The Hadoop distributed file system is used to synchronize and update the task protection status table across nodes, generating a data consistency verification table. Finally, based on the data consistency verification table, the ELK (Elasticsearch, Logstash, Kibana) stack is used to monitor the distributed system's behavior in real time, generating a behavior anomaly detection table. Based on the behavior anomaly detection table, machine learning models (such as random forests or support vector machines) are used to adaptively adjust the protection strategy and dynamically optimize security protection measures, thereby achieving comprehensive security protection for big data systems.

[0027] Preferably, step S1 includes the following steps:

[0028] Step S11: Obtain system operation data and perform sensitivity scanning and identification on the system operation data to obtain a data sensitivity classification table;

[0029] Specifically, a financial institution's risk control system can be used as a case study. The system's operational data includes customer transaction records, account balances, credit scores, and other information. The system's operational data is extracted from the database and imported into the Scikit-learn data preprocessing module for cleaning and standardization. The BERT model is used to extract embedding vectors from the text data, and matching analysis is performed using a predefined sensitive keyword library (such as "account balance" and "transaction amount"). Based on the matching results, the data is categorized into three levels: "high sensitivity," "medium sensitivity," and "low sensitivity," generating a data sensitivity classification table. For example, customer transaction amounts and account balances are marked as "high sensitivity," while information such as transaction time is marked as "medium sensitivity."

[0030] Step S12: Classify and summarize the system operation data according to the data sensitivity classification table to obtain a list of operation data types, and extract metadata from the system operation data according to the list of operation data types to obtain the initial dataset security metadata;

[0031] Specifically, in an Apache Spark distributed computing environment, the system's runtime data can be grouped and aggregated using its DataFrame API based on a data sensitivity grading table, grouping data of the same sensitivity level together. The Apache NiFi data flow management tool is then used to extract metadata for each group. By configuring the Apache NiFi ExtractText processor, metadata information such as data format, size, and source is extracted and stored as initial dataset security metadata. For example, for "highly sensitive" data, the metadata will include information such as encryption requirements and access permissions.

[0032] Step S13: Generate a globally unique identifier for the security metadata of the initial dataset to obtain a data identifier set, and mark the lifecycle of the system running data according to the data identifier set to obtain a data lifecycle table;

[0033] Specifically, the UUID (Universally Unique Identifier) ​​utility library in the Java programming language can be used to generate a unique identifier for each piece of data metadata in the initial dataset's security metadata, and this identifier can be stored as a set of data identifiers. The Apache Kafka message queue system is then used to mark the lifecycle of the data. In Apache Kafka, a topic is created for each data identifier to record lifecycle information such as creation time, access time, and update time, and this information is aggregated to generate a data lifecycle table. For example, the lifecycle information of a customer transaction record shows that it was created on January 1, 2024, last accessed on January 10, 2024, and is expected to expire on January 1, 2025.

[0034] Step S14: Map ownership of system running data according to the data lifecycle table to obtain the data ownership table, and bind the data security identifier set to the data ownership table to obtain the data security identifier set;

[0035] Specifically, the Apache Ranger access control tool can be used to map ownership of system runtime data. Based on information in the data lifecycle table, combined with system user and role information, data ownership is assigned to the appropriate user or department. For example, the ownership of a customer's transaction record data might be assigned to a specific analyst in the risk control department. Through Apache Ranger's policy engine, a data identifier set is associated and bound to the data ownership table to generate a data security identifier set. Specifically, access control policies are configured in Apache Ranger, binding data identifiers to owner information and setting corresponding access permissions. For example, for "highly sensitive" data, only the data owner and authorized auditors are allowed access; other personnel are not permitted to view it.

[0036] Step S15: Map the storage location of system operation data according to the data security identifier set to obtain a data location mapping table.

[0037] For a detailed implementation process of this embodiment, please refer to the sub-step of step S15.

[0038] This invention provides a basis for differentiated security strategies through data sensitivity grading, optimizing resource allocation and improving protection efficiency. It enriches the semantic information of data through classification, summarization, and metadata extraction, providing a structured foundation for security management and enhancing data traceability and controllability. It achieves full lifecycle tracking and management of data through globally unique identifier generation and lifecycle tagging, ensuring data security at different stages. It clarifies data ownership relationships through ownership mapping and identifier association binding mechanisms. Finally, it makes data storage management more orderly through storage location mapping, facilitating subsequent security protection operations.

[0039] Preferably, step S15 includes the following steps:

[0040] Step S151: Perform data flow statistics on the data security identifier set to obtain a data flow diagram, and identify the processing nodes of the data flow diagram to obtain a node processing relationship table;

[0041] Specifically, the Apache Flink stream processing framework can be used to perform data flow statistics on data security identifier sets. By deploying data monitoring tasks in Apache Flink, the transmission path and flow information of data between various processing nodes can be captured in real time. For example, in a distributed environment with multiple computing nodes, data flows from a data source node (such as a database) to a data cleaning node, and then to a data analysis node. Utilizing Apache Flink's state management and event-time processing capabilities, the flow path of each data identifier is recorded, generating a data flow graph. The Neo4j graph database is used to identify processing nodes in the data flow graph. By constructing a graph model in Neo4j, with each processing node as a node in the graph and the data flow as an edge, all processing nodes through which the data flows and their relationships are identified, generating a node processing relationship table. For example, the node processing relationship table records the complete path of data identifier "UUID-001" from node A to node B, and then to node C.

[0042] Step S152: Identify the dependencies of system operation data based on the node processing relationship table to obtain the data dependency matrix;

[0043] Specifically, Apache Spark's graph computing library GraphX ​​can be used to identify dependencies in the system's runtime data. By importing the node processing relationship table into GraphX, the PageRank and shortest path algorithms in its graph algorithm library can be used to analyze the dependencies between different processing nodes. For example, for the data identifier "UUID-001", by calculating its dependency strength among nodes A, B, and C, it is determined that the processing of the data at node B depends on the output of node A, while the processing at node C depends on the output of node B. A data dependency matrix is ​​generated based on these dependencies, where the rows and columns of the matrix represent processing nodes, and the values ​​in the matrix represent the dependency strength between nodes. For example, a value of 0.8 for node A to node B in the matrix indicates that node B has a high degree of dependency on node A.

[0044] Step S153: Construct a relationship graph based on the data dependency matrix to obtain a data flow tracing graph, and divide the data flow tracing graph into storage areas to obtain a storage area mapping table;

[0045] Specifically, the Neo4j graph database can be used to construct a data flow tracing graph in conjunction with a data dependency matrix. By using the dependencies in the data dependency matrix as edge weights and the processing nodes as nodes, a complete data flow tracing graph is constructed. For example, in the graph, the edge weight between node A and node B is 0.8, representing the degree of data dependency from node A to node B. Using Neo4j's graph partitioning algorithm, the data flow tracing graph is divided into storage areas based on the data flow path and the storage requirements of the nodes. For example, nodes processing sensitive data are assigned to high-security storage areas, while nodes processing ordinary data are assigned to ordinary storage areas. Finally, a storage area mapping table is generated, recording the storage area information corresponding to each processing node.

[0046] Step S154: Allocate address space for the storage area mapping table to obtain an address allocation table, and create a location index for the system running data based on the address allocation table to obtain a location index table;

[0047] Specifically, the Ceph distributed storage system can be used to allocate address space for storage region mapping tables. Through Ceph's dynamic object storage capabilities, an independent address space is allocated to each storage region. For example, an encrypted storage pool can be allocated to a high-security storage region, and a specific address range can be assigned to it. Using Ceph's object indexing capabilities, combined with the address allocation table, a location index is created for the system's operational data. For example, for data identifier "UUID-001", a location index record is created based on its storage location and address allocation information, including the storage pool where the data is stored, the object ID, and the address offset, ultimately generating a location index table.

[0048] Step S155: Perform location association mapping on the system running data according to the location index table to obtain the data location mapping table.

[0049] Specifically, the location index table can be managed and queried using the Elasticsearch search engine. By importing the location index table into Elasticsearch, its efficient full-text search and aggregate query capabilities can be leveraged to achieve location-based mapping of system data. For example, based on the location index information of the data identifier "UUID-001", a data location mapping record can be created in Elasticsearch, including information such as the data identifier, storage region, storage pool, object ID, and address offset. This ultimately generates a data location mapping table.

[0050] This invention achieves refined management and efficient, secure control of the data flow process through systematic analysis and mapping of data flow and processing nodes. By statistically analyzing data flow and identifying processing nodes, the flow path and processing logic of data within the system can be clearly presented. The identification and matrix representation of data dependencies further reveal the inherent connections between data, enabling the system to accurately identify mutual influences, thereby optimizing data processing workflows and reducing redundant computations and potential security risks. By constructing a data flow tracking graph and a storage area mapping table, the data flow status can be monitored in real time, ensuring the security and integrity of data during storage and transmission. Address space allocation and location index creation provide technical support for efficient data storage and rapid retrieval, improving the overall performance of the system.

[0051] Preferably, step S2 includes the following steps:

[0052] Step S21: Perform security area planning on the data location mapping table to obtain a security area division table;

[0053] Specifically, Ansible's automated configuration management tool can be used to plan security zones in a data location mapping table. Based on the sensitivity level and purpose of the data, data storage locations are divided into high-security, medium-security, and low-security zones. For example, high-security zones are used to store core business data (such as financial data), medium-security zones are used to store user behavior data, and low-security zones are used to store log data. Ansible playbook files define the storage location, access permissions, and encryption requirements for each security zone, and this information is then aggregated to generate a security zone partitioning table. For example, a high-security zone might be configured to use encrypted storage and restrict access permissions to a specific user group, while a low-security zone might allow broader access permissions but not enable encryption.

[0054] Step S22: Configure memory isolation according to the security zone partitioning table to obtain the physical memory isolation scheme table;

[0055] Specifically, Intel Software Guard Extensions (SGX) technology combined with Linux cgroups tools can be used to configure physical memory isolation. Based on a security zone partitioning table, independent memory regions are allocated to data with different security levels. For example, high-security data is allocated to an SGX-protected memory region, while medium- and low-security data are allocated to ordinary memory regions but are subject to resource restrictions and isolation via cgroups. The Linux cgroups memory controller sets memory usage limits and access permissions for each security region, and this configuration information is recorded in the physical memory isolation scheme table. For example, the high-security memory region is restricted to allowing only specific processes to access it, and its memory usage limit is 30% of the total system memory.

[0056] Step S23: Delineate partition boundaries based on the physical memory isolation scheme table to obtain the physical memory partition configuration table, and define access rules for the physical memory partition configuration table to obtain the access rule set;

[0057] Specifically, the Linux kernel's memory partitioning feature and the SELinux (Security-Enhanced Linux) security module can be used to define physical memory partition boundaries and access rules. Based on the physical memory isolation scheme table, the physical memory is divided into multiple independent memory partitions by modifying the Linux kernel's memory partitioning configuration file, with each partition corresponding to a security zone. For example, a high-security zone might allocate 16GB of memory, a medium-security zone 8GB, and a low-security zone the remaining memory. The access rules for each memory partition are defined using the SELinux policy language. For instance, a high-security zone might only allow specific processes to access it in read-only mode, a medium-security zone might allow read and write operations for relevant processes, but require authentication, and a low-security zone might allow broader access permissions but limit the maximum memory usage of a single process. These access rules are recorded in an access rule set.

[0058] Step S24: Construct the permission matrix based on the access rule set to obtain the memory access control table;

[0059] Specifically, the Apache Ranger permission management tool can be used in conjunction with Linux ACLs (Access Control Lists) to construct a permission matrix for the access rule set. Through Apache Ranger's policy management interface, the rules in the access rule set are converted into specific permission policies, assigning different access permissions to different memory partitions for each process or user group. For example, for the high-security memory partition, only system administrators and processes of specific business systems are allowed access, and access is restricted to read-only. For the medium-security memory partition, processes of relevant business systems are allowed to perform read and write operations, but authentication and authorization are required. The permission matrix records in detail the access permissions of each user or process to each memory partition, including read, write, and execute permission levels. Finally, this permission information is aggregated to generate a memory access control table.

[0060] Step S25: Perform real-time data encryption based on the memory access control table to obtain an encrypted data storage table.

[0061] For a detailed implementation process of this embodiment, please refer to the sub-step of step S25.

[0062] This invention significantly enhances data security and confidentiality in big data environments through systematic memory security management and data encryption measures. First, secure zone planning and memory isolation configuration enable partitioned management of physical memory, isolating data of different security levels in independent memory areas, effectively preventing unauthorized access and potential cross-contamination. Access rule definitions and permission matrix construction further refine memory access control, ensuring that only authorized operations can access specific memory areas, thereby enhancing the overall system security. Real-time data encryption provides an additional layer of protection for data in memory, making it difficult to decrypt and exploit even if data is illegally read from memory. This multi-layered protection mechanism, from memory isolation to access control to data encryption, not only effectively reduces the risk of data leakage during operation but also improves the system's resistance to attacks, making it particularly suitable for distributed computing scenarios with extremely high data security requirements.

[0063] Preferably, step S25 includes the following steps:

[0064] Step S251: Perform address space planning on the memory access control table to obtain the address space distribution map;

[0065] Specifically, the Linux kernel's memory management module can be used to perform address space planning for the memory access control table. By analyzing the system's physical memory layout and the permission information in the memory access control table, memory is divided into multiple address space regions, each corresponding to a different security level and access permissions. For example, high-security-level data is allocated to a separate address space region, while medium-security-level and low-security-level data are allocated to other regions. The system's memory layout information is obtained using the Linux / proc filesystem, and combined with the permission settings in the memory access control table, an address space distribution map is generated. This map records in detail the start address, end address, size, and corresponding access permissions of each address space region.

[0066] Step S252: Calculate the randomization parameters based on the address space distribution map to obtain the randomization parameter table;

[0067] Specifically, Address Space Layout Randomization (ASLR) technology can be used in conjunction with the Linux kernel's random number generator to randomize the address space. Based on information from the address space distribution map, a randomized offset for each address space region is calculated. For example, for high-security address space regions, the randomized offset range is set to 1GB; for medium-security and low-security regions, the randomized offset ranges are set to 512MB and 256MB, respectively. By calling the Linux kernel's `getrandom()` system call, a random offset is generated for each address space region, and these offsets are recorded in the randomization parameter table. The randomization parameter table records in detail the original address range and corresponding random offset for each address space region.

[0068] Step S253: Perform address mapping conversion on the memory access control table according to the randomization parameter table to obtain a dynamic address mapping table, and generate an encryption key table for the dynamic address mapping table to obtain an encryption key table;

[0069] Specifically, the Intel AES-NI (Advanced Encryption Standard New Instructions) instruction set can be used to perform address mapping and encryption key generation for the memory access control table. Based on the random offset in the randomization parameter table, addresses in the memory access control table are dynamically mapped. For example, for a high-security address space region, data at the original address 0x10000000 is mapped to the new address 0x30000000 after being converted by a random offset of 0x20000000. After the address mapping is completed, the Intel AES-NI instruction set is used to generate an encryption key corresponding to each dynamic address. The generation of encryption keys is based on the security level of the address space region and the random offset, ensuring that the encryption key for each dynamic address is unique and has high strength. The generated encryption keys are recorded in an encryption key table, which records each dynamic address and its corresponding encryption key in detail.

[0070] Step S254: Encrypt the dynamic address mapping table according to the encryption key table and perform integrity verification to obtain the encrypted data table and the data verification table;

[0071] Specifically, the data in the dynamic address mapping table can be encrypted using the OpenSSL encryption library, and integrity verification can be performed using the SHA-256 algorithm. Based on the encryption key in the encryption key table, the data block corresponding to each dynamic address is encrypted. For example, for the data block at dynamic address 0x30000000, AES-256-CBC encryption is performed using the corresponding encryption key. After encryption, an integrity check value is generated for the encrypted data block using the SHA-256 algorithm. The encrypted data block and its corresponding integrity check value are recorded in the encryption data table and the data check table, respectively. The encryption data table records each dynamic address and its encrypted data content in detail, while the data check table records the integrity check value for each encrypted data block.

[0072] Step S255: Update the storage location of the encrypted data table according to the data verification table to obtain the encrypted data storage table.

[0073] Specifically, the Ceph distributed storage system can be used to update the storage location of the encrypted data table. Based on the integrity check value in the data verification table, the encrypted data blocks are stored in Ceph's object storage. For example, for an encrypted data block with dynamic address 0x30000000, based on its integrity check value, it is stored in a specific Ceph storage pool, and its storage location information is recorded. Through Ceph's object storage functionality, a unique object ID is assigned to each encrypted data block, and the mapping relationship between the object ID, storage pool location, and dynamic address is recorded in the encrypted data storage table. The encrypted data storage table records detailed storage location information for the encrypted data block corresponding to each dynamic address.

[0074] This invention makes data storage locations unpredictable through address space planning and randomized parameter calculation, increasing the difficulty for attackers to obtain useful information. By combining dynamic address mapping and encryption keys, it provides more flexible and stronger encryption protection for data, ensuring that the storage and access of data in memory is difficult to crack. Simultaneously, the integrity verification mechanism after data encryption can effectively detect whether data has been tampered with during storage or transmission, further improving data reliability. Dynamic updates to storage locations make encrypted data management more flexible, better adapting to the needs of frequent data changes in distributed systems.

[0075] Preferably, the step of performing reference counting statistics on system memory data based on the encrypted data storage table specifically involves:

[0076] The system memory page table is obtained and traversed and analyzed to obtain memory page distribution data; wherein, the traversal and analysis adopts bitmap scanning technology to record the usage status of each memory page;

[0077] Specifically, the Linux ` / proc` filesystem and the `libhugetlbfs` tool can be used to obtain system memory page table information. By accessing the ` / proc / self / pagemap` file, the memory page table information of the current process can be obtained. Using the `pagemap` interface provided by `libhugetlbfs`, the bitmap information in the memory page table can be read, where each bit represents the status of a memory page (e.g., used, free, etc.). For example, for a region containing 1024 memory pages, the first bit in the bitmap being 1 indicates that the first memory page is used, and 0 indicates that it is unused. Through the above steps, memory page distribution data is generated, recording the usage status of each memory page.

[0078] The memory page distribution data is parsed using address space analysis to obtain memory space mapping data. Then, the data region is identified based on the memory space mapping data to obtain memory region marking data.

[0079] Specifically, the ` / proc / self / maps` file in Linux and the `nm` tool can be used to perform address space resolution on memory page distribution data. By reading the ` / proc / self / maps` file, the memory mapping information of the current process is obtained, including the start address, end address, and permissions of each memory segment. Combined with the memory page distribution data, the usage status of memory pages is associated with the memory mapping information to generate memory space mapping data. For example, a memory segment has a start address of 0x10000000, an end address of 0x10010000, and read / write permissions. Based on the memory page distribution data, it can be marked which pages within this memory segment are in use and which are free. Based on the memory space mapping data, data regions are identified. For example, memory pages located from 0x10000000 to 0x10001000 are marked as data region A, and memory pages located from 0x10002000 to 0x10003000 are marked as data region B. Finally, memory region marking data is generated.

[0080] Security attributes are extracted from the memory region marking data to obtain memory security feature data;

[0081] Specifically, the security attributes of memory region labeled data can be extracted using SELinux (Security-Enhanced Linux) and AppArmor security modules. By querying the SELinux security context and AppArmor security policy, the security attributes of each memory region are obtained. For example, for data region A, the SELinux security context is system_u:object_r:unlabeled_t:s0, indicating that this region is not explicitly labeled and poses a security risk; while for data region B, the security context is system_u:object_r:app_data_t:s0, indicating that this region is labeled as application data and has a higher security level. Simultaneously, the AppArmor security policy is used to further confirm the access permissions and security restrictions for each memory region. For example, the access permissions for data region B are restricted to read-only and can only be accessed by specific processes. Through these steps, the security attributes of each memory region are extracted, and memory security feature data is generated.

[0082] The memory security feature data is associated with the encrypted data storage table to obtain the data association mapping table, and the reference chain is traced on the data association mapping table to obtain the memory reference chain tracing data.

[0083] Specifically, an SQLite database can be used to store encrypted data storage tables and memory-safe feature data, and data association can be performed using SQL queries. The process involves matching data addresses in the encrypted data storage table with memory regions in the memory-safe feature data to generate a data association mapping table. For example, the encrypted data storage table records data block `data_block_1` stored at address 0x10001000, while the memory-safe feature data marks this address as belonging to data region A. Through association, the relationship between `data_block_1` and data region A is determined and recorded in the data association mapping table. The Linux `perf` tool is used to perform reference chain tracing on the data association mapping table. `perf`'s sampling function records the reference information for each data block. For example, `data_block_1` is referenced 3 times by process P1 and 2 times by process P2. This reference information is aggregated into the memory reference chain tracing data.

[0084] Based on the memory reference chain tracking data, the access frequency is statistically analyzed to obtain a data access frequency statistics table. The lifecycle of the data access frequency statistics table is then dynamically evaluated to obtain a data lifecycle evaluation table.

[0085] Specifically, the Prometheus monitoring system can be used to track memory reference chains and perform access frequency statistics. Using Prometheus's time-series database, the number of references and reference timestamps for each data block are recorded. For example, data_block_1 was referenced 5 times in the past 24 hours, with reference timestamps of 2024-06-01T08:00:00 and 2024-06-01T12:00:00, etc. Based on this data, a data access frequency statistics table is generated, recording the access frequency and time distribution of each data block. The Grafana visualization tool is used to dynamically evaluate the lifecycle of the data access frequency statistics table. Through Grafana's dashboard, data lifecycle evaluation rules are set; for example, if a data block has not been referenced in the past 7 days, it is marked as low priority; if it has been frequently referenced in the past 24 hours, it is marked as high priority. Based on these rules, a data lifecycle evaluation table is generated, recording the lifecycle status of each data block.

[0086] The citation count is updated based on the data lifecycle assessment table to obtain citation statistics. The citation statistics are then aggregated to obtain a data citation status table.

[0087] Specifically, a Python script can be used in conjunction with an SQLite database to update the reference count of the data lifecycle assessment table. The Python script reads the data lifecycle assessment table and updates the reference count of each data block based on its lifecycle status. For example, for a data block marked as high priority, `data_block_1`, its reference count increases by 1; for a data block marked as low priority, `data_block_2`, its reference count decreases by 1. The updated reference count is recorded in the reference count statistics. The reference count statistics are then aggregated to generate a data reference status table. For example, data blocks with a reference count greater than 5 are marked as "active," and data blocks with a reference count less than or equal to 0 are marked as "pending cleanup." Finally, the data reference status table records the reference status of each data block in detail.

[0088] This invention employs bitmap scanning to traverse and analyze memory page tables, enabling efficient and accurate acquisition of memory page usage status. Address space resolution and data region identification further clarify the storage locations and boundaries of different data in memory, enhancing data manageability. Security attribute extraction and data association mapping precisely link memory data with encrypted storage tables, allowing the system to track data reference chains in real time, thus achieving dynamic monitoring of data flow and usage. Access frequency statistics and dynamic lifecycle assessment further optimize data management strategies, enabling the system to flexibly adjust memory resource allocation and security measures based on actual data usage and lifecycle status. The data reference status table generated through reference counting updates and status aggregation provides the system with a comprehensive view of memory data status, helping the system promptly identify potential data leakage risks and providing a scientific basis for memory cleanup and optimization operations.

[0089] Preferably, the residual detection and analysis of the data reference status table specifically includes:

[0090] A memory scan of the data reference status table is performed to obtain a memory data distribution map.

[0091] Specifically, the Volatility tool can be used to perform a memory scan of the data reference status table. By using Volatility's pslist plugin to obtain memory usage data for all processes in the system and combining this information with the data reference status table, the memory regions associated with the target data block can be located. The Volatility malfind plugin is then used to scan these memory regions, detecting potential anomalous data remnants. During the scan, the tool generates a memory data distribution map, recording the address range, data type, and usage status of each memory region. For example, if the scan discovers unreferenced but not yet freed data blocks in a memory region, this information is recorded in the memory data distribution map.

[0092] Anomaly feature extraction is performed based on the memory data distribution map to obtain anomaly feature description data;

[0093] Specifically, the YARA rule engine, a malware detection tool, can be used to extract anomalous features from data in a memory data distribution map. YARA can match anomalous patterns in memory using predefined rules. Based on the memory data distribution map, the memory region is divided into multiple analysis units, and YARA rules are written for each unit. For example, a rule might be defined to detect unreferenced data blocks in memory that contain sensitive keywords (such as "password" or "key"). By running a YARA rule scan, anomalous feature description data is extracted, and the characteristic information of each anomalous data block, such as data content, location, and source, is recorded.

[0094] Residual rule matching is performed on the abnormal feature description data to obtain residual rule matching results; wherein, the residual rule matching adopts a preset expert system rule base;

[0095] Specifically, the rule matching engine in the Snort intrusion detection system can be used to perform residual rule matching on anomaly feature description data. Snort's rule matching engine can efficiently handle complex matching logic. An expert system rule base is pre-built, containing feature rules for various types of data residuals. For example, the rule base defines "unreferenced data blocks containing sensitive information" as high-risk residual data. Anomaly feature description data is input into Snort's rule matching engine, which matches it against the rules in the expert system rule base, generating residual rule matching results. The results record in detail whether each anomaly data block matches a residual rule in the rule base, as well as the specific rule number and description of the match.

[0096] Regions are labeled based on the residual rule matching results to obtain a data residual label set.

[0097] Specifically, a Python script can be used in conjunction with an SQLite database to process the residual rule matching results and generate a data residual tag set. The Python script reads the residual rule matching results and marks the memory regions containing anomalous data blocks based on the matching results. For example, if a data block in a memory region matches a high-risk residual rule, that region will be marked as a "high-risk residual region." The marking information is stored in the SQLite database, generating a data residual tag set. The data residual tag set records in detail the residual risk level, location, and related anomalous data block information for each memory region.

[0098] This invention achieves accurate identification and labeling of potential data remnants in memory through systematic residual detection and analysis. Memory scanning of the data reference status table provides a comprehensive understanding of the data distribution in memory. Anomaly feature extraction further filters out problematic data features, while residual rule matching based on an expert system rule base ensures the accuracy and reliability of the detection, effectively identifying residual data that does not conform to normal data lifecycles and usage patterns. Region labeling clearly identifies these potentially risky data.

[0099] Preferably, the step of clearing residual data through multi-level caching based on the data residual marker set specifically involves:

[0100] The residual data tag set is classified into hierarchical categories to obtain a residual data hierarchy table. Based on the residual data hierarchy table, the cache hierarchy is divided to obtain cache hierarchy structure data.

[0101] Specifically, Python scripts combined with an SQLite database can be used to classify and categorize residual data tags. Residual data is classified according to its risk level (e.g., high, medium, low risk) and data type (e.g., temporary data, sensitive data). For example, high-risk sensitive data is labeled as Level 1, medium-risk temporary data as Level 2, and low-risk ordinary data as Level 3. This classification information is stored in the SQLite database, generating a residual data classification table. Based on this classification table and the system's caching architecture, cache levels are determined. For example, the system might use a three-level cache architecture: L1 cache (CPU cache), L2 cache (memory cache), and L3 cache (disk cache). Based on data access frequency and importance, Level 1 data is allocated to the L1 cache, Level 2 data to the L2 cache, and Level 3 data to the L3 cache. The cache hierarchy structure data records the cache hierarchy information for each piece of residual data.

[0102] A cache cleanup strategy table is obtained by formulating a cleanup strategy for the cache hierarchy data.

[0103] Specifically, for detailed implementation process of this embodiment, please refer to the specific processing steps for formulating a cleanup strategy for cache hierarchy data.

[0104] The cleanup is performed according to the cache cleanup policy table, the cleanup execution status data is obtained, and the integrity of the cleanup execution status data is checked to obtain the memory cleanup verification table.

[0105] Specifically, Ansible automation tools can be used in conjunction with the Linux `dd` command and Python scripts for cleanup execution. Based on the policies in the cache cleanup policy table, Ansible calls the corresponding cleanup scripts to perform the cleanup operations. For example, for data in the L1 cache, the Linux `dd` command is used to zero out the data blocks; for data in the L2 cache, a Python script is used to archive the data to disk and delete the original data; for data in the L3 cache, cleanup operations are dynamically triggered based on system memory usage. During the cleanup execution, the status information of each cleanup operation is recorded, such as cleanup time, cleanup amount, and cleanup result, generating cleanup execution status data. The Linux `md5sum` command is used to perform integrity checks on the cleaned memory regions. For example, an MD5 checksum is generated for the cleaned memory region and compared with the checksum before cleanup to verify whether the cleanup operation was thorough and did not affect other data. Finally, the cleanup execution status data and integrity check results are summarized in a memory cleanup verification table, recording the cleanup verification information for each memory region.

[0106] This invention categorizes and classifies residual data tags, enabling differentiated cleanup strategies based on data importance and risk level. This avoids the waste and potential impact of a "one-size-fits-all" cleanup approach on system resources. The hierarchical structure of the cache further optimizes the targeting of cleanup operations, ensuring that the cleanup strategy adapts to the characteristics and needs of different cache levels. A post-cleanup integrity verification mechanism effectively verifies the cleanup results, ensuring that residual data is thoroughly removed while preventing accidental impacts on normal data.

[0107] Preferably, the step of formulating a cleanup strategy for the cache hierarchy data specifically involves:

[0108] Based on the identifier of each level in the cache hierarchy data, extract the data usage frequency, data importance, and data update time of the cache hierarchy data;

[0109] Specifically, for a detailed implementation process of this embodiment, please refer to the specific processing steps for extracting the data usage frequency, data importance, and data update time of the cache hierarchy structure data based on each level identifier in the cache hierarchy structure data.

[0110] If the data is used less than once a day, has low importance, and has not been updated for more than 48 hours, then the cleanup priority is set to 1, and the data is deleted directly, thus obtaining the first cache cleanup strategy;

[0111] Specifically, for data used less than once a day, of low importance, and not updated for more than 48 hours, a direct deletion cleanup strategy is adopted. This strategy is implemented using the Ansible automation tool in conjunction with a Python script. The specific operation is as follows: Ansible calls the Python script, which queries the Prometheus database to filter out data blocks that meet the criteria, and then calls the Linux `rm` command or a memory cleanup tool (such as `echo 3> / proc / sys / vm / drop_caches`) to directly delete these data blocks. After cleanup, the cleanup operation is logged, including the identifier of the deleted data block, the cleanup time, etc., and these logs are stored in an SQLite database, forming the execution phase of the first cache cleanup strategy.

[0112] If the data is used 1-5 times a day, is of medium importance, and has not been updated for more than 72 hours, then set the cleanup priority to 3, use the archiving method and retain 7 days of backup, thus obtaining the second cache cleanup strategy;

[0113] Specifically, for data used 1-5 times per day, of medium importance, and not updated for more than 72 hours, a cleanup strategy of archiving and retaining a 7-day backup is adopted. The data archiving operation is implemented using the Rsync tool in conjunction with a Cron scheduled task. Specifically, Rsync copies eligible data blocks from the cache to a backup storage location (such as NFS shared storage or a backup directory on the local disk), setting the retention period to 7 days. After archiving is complete, an Ansible script deletes the data blocks from the original cache and logs the archiving and deletion operations. These logs are also stored in an SQLite database, forming the execution phase of the second cache cleanup strategy.

[0114] If the data is used more than 5 times a day or is of high importance, the cleaning priority is set to 5. The system memory usage rate is obtained. The cleaning operation is triggered only when the system memory usage rate exceeds the preset threshold and the duration exceeds the preset observation period. During the cleaning, the data with the earliest access time is selected first for cleaning, and the amount of data cleaned each time does not exceed 5% of the total system memory.

[0115] Specifically, for data used more than 5 times per day or of high importance, a dynamic triggering cleanup strategy is adopted. The Prometheus monitoring system is used to collect system memory usage in real time, and a high-water mark threshold (e.g., 80%) and observation period (e.g., 15 minutes) for memory usage are set via Alertmanager. When system memory usage exceeds the threshold and the duration exceeds the observation period, a cleanup operation is triggered. During cleanup, data with the earliest most recent access time is prioritized for cleanup, and the amount of data cleaned each time does not exceed 5% of the total system memory. This strategy is implemented using a Python script combined with the Linux `free` command. The script filters data blocks that meet the criteria based on Prometheus alert information and calls the `free` command to release memory. The logs of the cleanup operation are also stored in an SQLite database, forming the execution link of the third cache cleanup strategy.

[0116] The system load is periodically predicted using a preset prediction model to obtain the predicted load value. When the predicted load value exceeds the preset warning threshold, the time threshold in the second and third cache cleanup strategies is dynamically adjusted, that is, the waiting time of 72 hours is adjusted to 36 hours.

[0117] Specifically, Prometheus's predictive models (such as linear regression models based on time-series data) can be used to periodically predict system load. When the predicted load exceeds a preset warning threshold (e.g., 90%), the time thresholds in the second and third cache cleanup strategies are dynamically adjusted using Ansible automation tools. For example, the 72-hour wait time in the second cache cleanup strategy can be adjusted to 36 hours. The Ansible script updates the cleanup strategy configuration file based on the prediction results and reloads the cleanup task. The adjusted cleanup strategy configuration information is also recorded in the SQLite database.

[0118] The first cache cleanup strategy, the second cache cleanup strategy, and the third cache cleanup strategy are denoted as the cache cleanup strategy table.

[0119] Specifically, the detailed information of the three cache cleanup strategies mentioned above (including cleanup conditions, priorities, cleanup methods, and log records) is finally summarized in an SQLite database to generate a cache cleanup strategy table. A Python script periodically queries the database to generate a report on the cache cleanup strategies, displaying the current cleanup strategy configuration, execution status, and cleanup effects.

[0120] This invention, through a comprehensive evaluation of the usage frequency, importance, and update time of cache hierarchy data, can formulate differentiated cleanup strategies based on the characteristics of different data, ensuring that the cleanup operation is both efficient and meets the actual needs of system operation. The direct deletion strategy for low-priority data can quickly release memory resources, while the archiving and backup strategy for medium-priority data preserves data traceability during cleanup, preventing the loss of important data. The conditional cleanup strategy for high-priority data fully considers system load and memory usage, avoiding negative impacts on system performance due to cleanup operations. Through predictive models for dynamic monitoring of system load and flexible adjustment of time thresholds, the cleanup strategy can be optimized in real time based on the system's operating status, further improving the system's resource utilization efficiency and stability.

[0121] Preferably, the step of extracting the data usage frequency, data importance, and data update time of the cache hierarchy data based on each hierarchy identifier in the cache hierarchy data specifically involves:

[0122] Based on the hierarchical identifier, obtain the data access time distribution characteristics of the hierarchy, and calculate the time decay weight coefficient λ. Where t is the time interval between the data update time and the current time, and T is the feature time constant, which is obtained by performing exponential decay fitting on the historical access data of the hierarchy;

[0123] Specifically, the Prometheus monitoring system can be used to record the access timestamps of each data object in the cache hierarchy. Using Prometheus's time-series database, access timestamps for each data object over the past 30 days are collected and stored as time-series data. These timestamps are then analyzed using Grafana's visualization tools to generate a data access time distribution map. An exponential decay fitting is performed using a Python script combined with the NumPy and SciPy libraries. The script reads the time-series data from Prometheus, calculates the time interval t between each timestamp and the current time, and applies the decay weights according to the formula. Calculate the time decay weighting coefficient, where T is the characteristic time constant. For example, for a data object whose last access time was 10 hours ago, and the characteristic time constant T is set to 24 hours, its time decay weighting coefficient is: These weighting coefficients are stored in an SQLite database. A Python script, combined with the curve fitting function in the SciPy library, is used to perform exponential decay fitting on the historical access data of each level. The script reads the historical access timestamps recorded in Prometheus and converts them into a time interval sequence. The SciPy curve_fit function is then used to perform exponential decay fitting on the time interval sequence, with the fitting formula being: Where y represents the access frequency at time t, which is the target value of the formula, i.e., the current access frequency that we hope to predict or fit. It reflects the frequency with which the data object is accessed at a specific point in time. A is the initial access frequency, which is the access frequency at time t = 0, i.e., the frequency when the data object was last accessed. It is a constant representing the "popularity" or "activity" of the data object at the time of the last access. If a data object has been accessed 10 times in the past hour, then A can be approximated as 10. t represents the length of time from the last access to the present. This variable reflects the time elapsed since the last access to the data object. Over time, the access frequency of the data object usually decreases. This decrease can be described by exponential decay. If the current time is 2024-06-01T12:00:00, and the last access time of the data object was 2024-05-31T12:00:00, then t = 24 hours. T is the characteristic time constant, which represents the time required for the data access frequency to decay to 1 / e (approximately 36.8%) of the initial value A. It reflects the rate at which data access frequency decays over time. If T = 48 hours, this means that the access frequency of a data object will decay to approximately 36.8% of its initial value after 48 hours. The characteristic time constant T for each level is obtained by fitting a large amount of historical data. For example, for a cache level, the fitting result shows that its characteristic time constant T is 48 hours. This value reflects the decay characteristics of the access frequency of data at that level over time and is stored in the SQLite database.

[0124] Multiply the time decay weighting coefficient by the original number of accesses to obtain the data usage frequency; where the original number of accesses is the total number of times each data object at the record level is accessed within the preset time period;

[0125] Specifically, a Python script can be used in conjunction with an SQLite database to calculate the usage frequency of data. The script reads the raw access counts of each data object from Prometheus over the past 24 hours and the corresponding time decay weighting coefficient from the SQLite database. By multiplying the time decay weighting coefficient by the raw access count, the weighted access count for each data object is obtained. For example, if a data object was accessed 10 times in the past 24 hours, its time decay weighting coefficient is 0.61, then its weighted access count is 10 × 0.61 = 6.1. The script then sums the weighted access counts of all data objects to calculate the average weighted access count for each level, i.e., the data usage frequency.

[0126] Data service tags for each level are obtained based on the level identifier. These tags include core business data, general business data, and temporary data. The importance of core business data is set to high, the importance of general business data is set to medium, and the importance of temporary data is set to low.

[0127] Specifically, a Python script can be used in conjunction with an SQLite database to categorize data objects using business tags and assign them importance. The script reads the business tag information for each data object, which is pre-stored in the SQLite database. This tag information includes core business data, general business data, and temporary data. Based on the business tags, the script sets the importance of core business data to high, general business data to medium, and temporary data to low. For example, a data object labeled "core business data" would have an importance of 3. This importance information is stored in the SQLite database and correlated with other information about the data object (such as usage frequency and update time).

[0128] Data importance is categorized into three levels: high, medium, and low. The data update time records the timestamp of the last data modification.

[0129] Specifically, the Prometheus monitoring system can be used to record the last update timestamp for each data object. The `time()` function in Prometheus is used to obtain the last modification time of each data object and store it as time-series data. Simultaneously, a Python script reads the importance information of the data objects from an SQLite database and associates it with the update timestamp. For example, a data object with a medium importance might have its last update time as 2024-06-01T12:00:00. The script aggregates this information and stores it in the SQLite database, generating a complete data table containing data usage frequency, importance, and update time.

[0130] This invention achieves refined evaluation and management of cached data by introducing a time decay weighting coefficient and data service tags. The time decay weighting coefficient dynamically adjusts the data usage frequency based on the data access time distribution characteristics, enabling the system to more accurately reflect the current importance of data and avoid overestimating or underestimating data value due to the accumulation of historical access counts. Simultaneously, classifying data importance based on data service tags further clarifies the value and priority of different types of data in the system, providing a higher level of protection for core business data, while allowing for more aggressive cleanup strategies for temporary data. This evaluation method, combining time and business attributes, not only improves the scientific rigor and flexibility of cache cleanup strategies but also dynamically adjusts data management methods according to the actual needs of system operation, ensuring efficient system operation while maximizing the security and availability of critical data.

[0131] Therefore, the embodiments should be considered as exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of the equivalents of the application are intended to be included within the invention.

[0132] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features of the invention herein.

Claims

1. A method for protecting the security of big data, characterized in that, Includes the following steps: Step S1: Obtain system operation data; map the storage location of the system operation data to obtain a data location mapping table; Step S2: Perform real-time data encryption based on the data location mapping table to obtain an encrypted data storage table; Step S3: Obtain system memory data; perform reference counting statistics on the system memory data based on the encrypted data storage table to obtain a data reference status table; perform residual detection analysis on the data reference status table to obtain a data residual tag set; perform multi-level cache clearing of residual data based on the data residual tag set to obtain a memory cleanup verification table; The memory cleanup verification table is defragmented and optimized to obtain an optimized memory layout diagram; specifically, the reference counting statistics of system memory data based on the encrypted data storage table are performed as follows: The system memory page table is obtained and traversed and analyzed to obtain memory page distribution data; wherein, the traversal and analysis adopts bitmap scanning technology to record the usage status of each memory page; The memory page distribution data is parsed using address space analysis to obtain memory space mapping data. Then, the data region is identified based on the memory space mapping data to obtain memory region marking data. Security attributes are extracted from the memory region marking data to obtain memory security feature data; The memory security feature data is associated with the encrypted data storage table to obtain the data association mapping table, and the reference chain is traced on the data association mapping table to obtain the memory reference chain tracing data. Based on the memory reference chain tracking data, the access frequency is statistically analyzed to obtain a data access frequency statistics table. The lifecycle of the data access frequency statistics table is then dynamically evaluated to obtain a data lifecycle evaluation table. The citation count is updated based on the data lifecycle assessment table to obtain citation count statistics. The citation count statistics are then aggregated to obtain a data citation status table. Specifically, the step of clearing residual data through a multi-level cache based on a data residual tag set includes: The residual data tag set is classified into hierarchical categories to obtain a residual data hierarchy table. Based on the residual data hierarchy table, the cache hierarchy is divided to obtain cache hierarchy structure data. A cache cleanup strategy table is obtained by formulating a cleanup strategy for the cache hierarchy data. The cleanup is performed according to the cache cleanup policy table, the cleanup execution status data is obtained, and the integrity of the cleanup execution status data is checked to obtain the memory cleanup verification table. Step S4: Based on the optimized memory layout diagram, construct a secure channel for the distributed nodes to obtain a node communication configuration table; perform task scheduling and allocation on the node communication configuration table to obtain a secure task allocation table; perform interrupted task data protection processing based on the secure task allocation table to obtain a task protection status table; perform node synchronization updates on the task protection status table to obtain a data consistency verification table. Step S5: Perform real-time behavior monitoring of the distributed system based on the data consistency verification table to obtain a behavior anomaly detection table; adaptively adjust the protection strategy of the distributed system according to the behavior anomaly detection table to perform big data security protection.

2. The big data security protection method according to claim 1, characterized in that, Step S1 includes the following steps: Step S11: Obtain system operation data and perform sensitivity scanning and identification on the system operation data to obtain a data sensitivity classification table; Step S12: Classify and summarize the system operation data according to the data sensitivity classification table to obtain a list of operation data types, and extract metadata from the system operation data according to the list of operation data types to obtain the initial dataset security metadata; Step S13: Generate a globally unique identifier for the security metadata of the initial dataset to obtain a data identifier set, and mark the lifecycle of the system running data according to the data identifier set to obtain a data lifecycle table; Step S14: Map ownership of system running data according to the data lifecycle table to obtain the data ownership table, and bind the data security identifier set to the data ownership table to obtain the data security identifier set; Step S15: Map the storage location of system operation data according to the data security identifier set to obtain a data location mapping table.

3. The big data security protection method according to claim 2, characterized in that, Step S15 includes the following steps: Step S151: Perform data flow statistics on the data security identifier set to obtain a data flow diagram, and identify the processing nodes of the data flow diagram to obtain a node processing relationship table; Step S152: Identify the dependencies of system operation data based on the node processing relationship table to obtain the data dependency matrix; Step S153: Construct a relationship graph based on the data dependency matrix to obtain a data flow tracing graph, and divide the data flow tracing graph into storage areas to obtain a storage area mapping table; Step S154: Allocate address space for the storage area mapping table to obtain an address allocation table, and create a location index for the system running data based on the address allocation table to obtain a location index table; Step S155: Perform location association mapping on the system running data according to the location index table to obtain the data location mapping table.

4. The big data security protection method according to claim 1, characterized in that, Step S2 includes the following steps: Step S21: Perform security area planning on the data location mapping table to obtain a security area division table; Step S22: Configure memory isolation according to the security zone partitioning table to obtain the physical memory isolation scheme table; Step S23: Delineate partition boundaries based on the physical memory isolation scheme table to obtain the physical memory partition configuration table, and define access rules for the physical memory partition configuration table to obtain the access rule set; Step S24: Construct the permission matrix based on the access rule set to obtain the memory access control table; Step S25: Perform real-time data encryption based on the memory access control table to obtain an encrypted data storage table.

5. The big data security protection method according to claim 4, characterized in that, Step S25 includes the following steps: Step S251: Perform address space planning on the memory access control table to obtain the address space distribution map; Step S252: Calculate the randomization parameters based on the address space distribution map to obtain the randomization parameter table; Step S253: Perform address mapping conversion on the memory access control table according to the randomization parameter table to obtain a dynamic address mapping table, and generate an encryption key table for the dynamic address mapping table to obtain an encryption key table; Step S254: Encrypt the dynamic address mapping table according to the encryption key table and perform integrity verification to obtain the encrypted data table and the data verification table; Step S255: Update the storage location of the encrypted data table according to the data verification table to obtain the encrypted data storage table.

6. The big data security protection method according to claim 1, characterized in that, The residual detection and analysis of the data reference status table specifically includes: A memory scan of the data reference status table is performed to obtain a memory data distribution map; Anomaly feature extraction is performed based on the memory data distribution map to obtain anomaly feature description data; Residual rule matching is performed on the abnormal feature description data to obtain residual rule matching results; wherein, the residual rule matching adopts a preset expert system rule base; Regions are labeled based on the residual rule matching results to obtain a data residual label set.

7. The big data security protection method according to claim 1, characterized in that, The specific strategy for cleaning up cache hierarchy data is as follows: Based on the identifier of each level in the cache hierarchy data, extract the data usage frequency, data importance, and data update time of the cache hierarchy data; If the data is used less than once a day, has low importance, and has not been updated for more than 48 hours, then the cleanup priority is set to 1, and the data is deleted directly, thus obtaining the first cache cleanup strategy; If the data is used 1-5 times a day, is of medium importance, and has not been updated for more than 72 hours, then set the cleanup priority to 3, use the archiving method and retain 7 days of backup, thus obtaining the second cache cleanup strategy; If the data is used more than 5 times a day or is of high importance, the cleaning priority is set to 5. The system memory usage rate is obtained. The cleaning operation is triggered only when the system memory usage rate exceeds the preset threshold and the duration exceeds the preset observation period. During the cleaning, the data with the earliest access time is selected first for cleaning, and the amount of data cleaned each time does not exceed 5% of the total system memory. The system load is periodically predicted using a preset prediction model to obtain the predicted load value. When the predicted load value exceeds the preset warning threshold, the time threshold in the second and third cache cleanup strategies is dynamically adjusted, that is, the waiting time of 72 hours is adjusted to 36 hours. The first cache cleanup strategy, the second cache cleanup strategy, and the third cache cleanup strategy are denoted as the cache cleanup strategy table.

8. The big data security protection method according to claim 7, characterized in that, The step of extracting the data usage frequency, data importance, and data update time of the cache hierarchy data based on each hierarchy identifier in the cache hierarchy data specifically involves: Based on the hierarchical identifier, obtain the data access time distribution characteristics of the hierarchy, and calculate the time decay weight coefficient λ. Where t is the time interval between the data update time and the current time, and T is the feature time constant, which is obtained by performing exponential decay fitting on the historical access data of the hierarchy; Multiply the time decay weighting coefficient by the original number of accesses to obtain the data usage frequency; where the original number of accesses is the total number of times each data object at the record level is accessed within the preset time period; Data service tags for each level are obtained based on the level identifier. These tags include core business data, general business data, and temporary data. The importance of core business data is set to high, the importance of general business data is set to medium, and the importance of temporary data is set to low. Data importance is categorized into three levels: high, medium, and low. The data update time records the timestamp of the last data modification.

Images