Audit data security management and control system based on big data

Abstract

The invention discloses an auditing data security management and control system based on big data, and relates to the field of data management and control, and the system comprises a distributed crawler which collects auditing logs, operation records and network traffic in real time. And the stream batch integrated engine performs metadata alignment and timestamp synchronization on heterogeneous data to generate a unified audit event stream. The intelligent desensitization module carries out gradient desensitization on the sensitive fields and constructs reversible blood relationship mapping. According to the hierarchical storage, data are stored in an encrypted memory pool, an SSD and a cold standby cluster according to data sensitivity and access frequency, and consistent hash load balancing is achieved. Multi-modal safety monitoring analyzes data in real time through a three-level assembly line. And the hierarchical response module writes a permission chain according to the risk trigger action and the key audit event hash value, and constructs a tamper-proof evidence chain. The method has the advantages that the data security, privacy protection and efficient emergency response are ensured through intelligent desensitization, hierarchical storage and security analysis of the audit data, and the non-tampering property of the data is ensured by utilizing the block chain technology.

Description

Technical Field

[0001] This invention relates to the field of data management, and in particular to an audit data security management system based on big data. Background Technology

[0002] With the rapid development of information technology, big data is being applied more and more widely across various industries, especially in the field of auditing. The introduction of big data technology has provided auditing work with more efficient and accurate analytical tools. However, this has also led to increasingly serious issues related to audit data security. Audit data often involves a large amount of sensitive information; once leaked or tampered with, it can not only have a significant impact on an organization's reputation but may also lead to legal liabilities. Therefore, ensuring the security, integrity, and confidentiality of audit data has become a focus of attention for all sectors.

[0003] Current audit data security management systems on the market fail to employ efficient encryption and compression mechanisms during data collection and transmission, resulting in poor data security during transmission and storage. Secondly, existing systems have weak capabilities in handling heterogeneous data, often struggling to achieve efficient integration and unified analysis between different data sources. They lack key functions such as timestamp synchronization and metadata alignment, leading to compromised data consistency and timeliness. Furthermore, many systems lack intelligent de-identification and lineage tracing functions, making it difficult to accurately protect sensitive data and ensuring data traceability, increasing the risk of data leakage and misuse. Regarding storage strategies, most systems fail to reasonably classify and dynamically adjust data based on sensitivity levels and access frequency, resulting in low storage efficiency and increased security vulnerabilities. In terms of security monitoring, many systems rely solely on basic rule detection, lacking multi-dimensional anomaly detection based on advanced technologies such as deep learning and graph computing, resulting in weak ability to identify complex security threats. Finally, many systems fail to fully utilize blockchain technology to ensure data immutability and notarization, posing risks to the legal compliance of audit data. Summary of the Invention

[0004] To improve the existing system, a big data-based audit data security management system is provided. This system uses big data technology to achieve real-time monitoring, intelligent de-identification, layered storage, and multi-dimensional security analysis of audit data, ensuring data security, privacy protection, and efficient emergency response. At the same time, blockchain technology is used to ensure the immutability and legal validity of the data.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A big data-based audit data security management system includes: Distributed crawler collection module: The module is deployed on the nodes of the audit data source and collects structured audit logs, unstructured operation records and raw network traffic packets in real time. After compression and encryption through the data channel, it is transmitted to the stream and batch processing engine module. Integrated batch and stream processing engine module: This module performs metadata alignment and timestamp synchronization on heterogeneous data to generate audit event stream data with unified spatiotemporal dimensions; Intelligent desensitization and lineage management module: The module identifies sensitive fields in the audit data, performs gradient desensitization on the audit data, and builds a reversible mapping relationship chain between the desensitized data and the original data by adding data lineage tags; Tiered storage module: Based on data sensitivity level and access frequency, the module stores data in layers in an encrypted memory pool, an SSD hot storage area, and a cold backup cluster, and performs load balancing processing on data shards through a consistent hashing algorithm; Multimodal security monitoring module: The module monitors audit data in real time by constructing a parallel three-level processing pipeline, including a stream processing layer, a graph computing layer, and a deep analysis layer; Tiered response and blockchain evidence storage module: The module triggers a multi-level response mechanism based on the data risk level, including real-time session blocking, dynamic permission downgrading, and audit mark tracing, and writes the hash value of key audit events into the permissioned chain to build an tamper-proof evidence chain.

[0006] Preferably, the distributed crawler acquisition module specifically includes: Structured data unit: The unit parses the real-time log stream by event type and batch maps and parses historical data by field; Unstructured data unit: The unit unifies and slices unstructured data, and performs semantic parsing and behavioral intent recognition through OCR text recognition; Network traffic unit: The unit is used to extract key fields such as IP, port, URL, and file transfer content, and to annotate the data packet time series and transmission path; Cross-system data pool unit: The unit stores data of different formats from different systems through a distributed database, and constructs a cross-system audit data pool for data classification and data synchronization; Compression and Encryption Unit: The unit compresses the data using a compression algorithm and encrypts the data using an encryption algorithm, then transmits the processed data to the batch processing engine module.

[0007] Preferably, the integrated batch processing engine module specifically includes: Metadata Alignment Unit: This unit unifies field names, types, and data granularity through the mapping relationship between heterogeneous data fields and unified audit fields, and adds metadata tags to data from different sources; Timestamp synchronization unit: The unit extracts the original timestamp of each data, aligns the data based on the original timestamp of the event, and controls the delay window of out-of-order data through Flink's Watermark mechanism; Event Stream Data Processing Unit: This unit converts events from different sources into audit events, aggregates and serializes cross-system events, and generates audit event stream data with global sequence numbers.

[0008] Preferably, the intelligent desensitization and bloodline management module specifically includes: Sensitive Field Identification Unit: The unit identifies sensitive fields in audit data through a pre-trained NLP model and extracts sensitive information entities from the data through named entity recognition technology; Gradient desensitization unit: The unit reduces the accuracy of data based on the sensitivity of sensitive fields, and uses different desensitization techniques for each sensitive field, including encryption desensitization, masking desensitization, data generalization and data obfuscation; Data lineage marker: During the desensitization process, the unit adds a lineage marker to each piece of data, including the data source identifier, desensitization type, and data version number; Reversible mapping relationship chain unit: The unit is used to record the mapping relationship between desensitized data and original data and establish a mapping table, and restore the desensitized data to the original data through a reversible desensitization mechanism.

[0009] Preferably, the hierarchical storage module specifically includes: Sensitivity Level Classification Unit: The unit classifies data into high-sensitivity data, medium-sensitivity data, low-sensitivity data, high-frequency data, medium-frequency data, and low-frequency data based on data sensitivity and data access frequency; Tiered storage unit: The unit classifies and stores data into an encrypted memory pool, an SSD hot storage area, and a cold backup cluster based on the characteristics of the partitioned data; Load balancing unit: The unit distributes data of different sensitivity levels through a consistent hashing algorithm, reduces data redistribution by increasing the number of virtual nodes, and avoids storage overload by periodically and dynamically adjusting the data hash distribution.

[0010] Preferably, the multimodal safety monitoring module specifically includes: Stream processing layer unit: The unit includes a real-time metric deviation monitor based on a CUSUM control chart; Graph computation layer unit: The unit includes a dynamic temporal graph network modeler; Deep analysis layer unit: The unit includes a joint anomaly detection model that integrates a Transformer temporal encoder and a GNN graph neural network.

[0011] Preferably, the stream processing layer unit specifically includes: Key monitoring metrics are extracted from the audit event stream, including user access frequency, operation duration, session latency, system call frequency, number of network access packets, number of login failures, and frequency of sensitive operations. By calculating the mean and variance through historical statistical modeling, a behavioral baseline for each type of operator is established, while real-time indicators are standardized. The CUSUM algorithm continuously tracks indicator deviations, and when the accumulated deviations exceed a preset threshold, a deviation alarm is triggered.

[0012] Preferably, the graph computation layer unit specifically includes: The entities in the audit data are mapped to nodes in the graph, and the operation behaviors are constructed as edges with timestamps, forming a dynamic time sequence graph structure, with each edge carrying behavior attributes; Extract the interaction frequency, path depth, and behavioral pattern graph structure features of nodes to identify abnormal relationship patterns; Track the evolution trend of the graph over time, detect abrupt changes, update the graph structure in real time, and obtain the current system status.

[0013] Preferably, the deep analysis layer unit specifically includes: Data from the stream processing layer and the graph computation layer is fused to obtain time series data, and the Transformer encoder is used to capture long-term dependencies and temporal patterns in the time series data. A joint model is trained by using self-supervised learning and contrastive learning to map normal and abnormal behaviors into distinguishable feature representations. Input the real-time behavioral sequence and graph structure of the audit data into the model, and output the risk score and anomaly type label for each behavioral entity.

[0014] Preferably, the hierarchical response and blockchain evidence storage module specifically includes: Tiered Response Unit: This unit defines corresponding levels, including mild, moderate, and high responses, based on real-time response tiers according to event classifications of various audit data. Blockchain Evidence Storage Unit: The unit stores key audit data, abnormal events, and response results in the blockchain, generates a fixed-length hash value for the event data through a hash algorithm, and performs automated evidence storage through a smart contract.

[0015] Compared with the prior art, the advantages of the present invention are: The system efficiently collects various types of audit data through a distributed crawler module and ensures data security through encrypted transmission. The integrated stream and batch processing engine module effectively integrates heterogeneous data through metadata alignment and timestamp synchronization, ensuring data consistency and timeliness. The intelligent desensitization and lineage management module achieves data privacy protection and traceability through gradient desensitization and lineage tracing. Simultaneously, the tiered storage module dynamically adjusts storage strategies based on data sensitivity and access frequency, optimizing storage performance and security. The multimodal security monitoring module combines stream processing, graph computing, and deep analytics technologies to perform multi-dimensional anomaly detection on audit data, effectively identifying potential security threats. The tiered response and blockchain evidence storage module triggers corresponding emergency responses based on risk levels and ensures the immutability and legal validity of audit data through blockchain technology. The overall system not only ensures data security but also improves audit efficiency and response speed, ensuring precise and controllable data governance, and providing strong audit protection for enterprises and institutions. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the system proposed in this invention; Figure 2 This is a diagram of the distributed crawler data collection module proposed in this invention; Figure 3 This is a block diagram of the batch processing engine module proposed in this invention; Figure 4 This is a diagram of the intelligent desensitization and bloodline management module proposed in this invention; Figure 5 This is a diagram of the hierarchical storage module proposed in this invention; Figure 6 This is a diagram of the multimodal safety monitoring module proposed in this invention; Figure 7 This is a diagram of the hierarchical response and blockchain evidence storage module proposed in this invention. Detailed Implementation

[0017] The following description is intended to disclose the invention and enable those skilled in the art to implement it. The preferred embodiments described below are merely examples, and other obvious variations will occur to those skilled in the art.

[0018] See Figure 1 As shown, the big data-based audit data security management system includes: Distributed crawler collection module: The module is deployed on the nodes of the audit data source and collects structured audit logs, unstructured operation records and raw network traffic packets in real time. After compression and encryption through the data channel, it is transmitted to the stream and batch processing engine module. Integrated batch and stream processing engine module: This module performs metadata alignment and timestamp synchronization on heterogeneous data to generate audit event stream data with unified spatiotemporal dimensions; Intelligent desensitization and lineage management module: The module identifies sensitive fields in the audit data, performs gradient desensitization on the audit data, and builds a reversible mapping relationship chain between the desensitized data and the original data by adding data lineage tags; Tiered storage module: Based on data sensitivity level and access frequency, the module stores data in layers in an encrypted memory pool, an SSD hot storage area, and a cold backup cluster, and performs load balancing processing on data shards through a consistent hashing algorithm; Multimodal security monitoring module: The module monitors audit data in real time by constructing a parallel three-level processing pipeline, including a stream processing layer, a graph computing layer, and a deep analysis layer; Tiered response and blockchain evidence storage module: The module triggers a multi-level response mechanism based on the data risk level, including real-time session blocking, dynamic permission downgrading, and audit mark tracing, and writes the hash value of key audit events into the permissioned chain to build an tamper-proof evidence chain.

[0019] See Figure 2 As shown, the distributed crawler acquisition module specifically includes: Structured data unit: The unit parses the real-time log stream by event type and batch maps and parses historical data by field; Unstructured data unit: The unit unifies and slices unstructured data, and performs semantic parsing and behavioral intent recognition through OCR text recognition; Network traffic unit: The unit is used to extract key fields such as IP, port, URL, and file transfer content, and to annotate the data packet time series and transmission path; Cross-system data pool unit: The unit stores data of different formats from different systems through a distributed database, and constructs a cross-system audit data pool for data classification and data synchronization; Compression and Encryption Unit: The unit compresses the data using a compression algorithm and encrypts the data using an encryption algorithm, then transmits the processed data to the batch processing engine module.

[0020] Specifically, the real-time log stream is categorized by event type, parsed into structured data using field separators, and then grouped by event type, such as login events and request events. Historical data is then mapped in batches according to predefined field mapping rules. Unstructured data of different formats, such as images, audio, video, and text, are uniformly converted into a processable format. For image data, image recognition technology is used to convert it into text information. Large blocks of unstructured data are divided into smaller data segments for processing, such as dividing long text into several paragraphs or cutting videos into frames. OCR technology is used to recognize text in images, and natural language processing technology is used for semantic parsing to identify behavioral intentions. Extract key fields from network traffic and label the time sequence and transmission path of data packets. Key fields such as IP address, port, URL and file transfer content are extracted from network data packets. The timestamp of each data packet is labeled to form the time sequence of data packets. The transmission path of each data packet is recorded according to routing information or other identifiers. The cross-system data pool uses a distributed database to store data from different systems. Each system has a different storage format. A data synchronization mechanism ensures that the data is updated in real time. The data is classified and managed according to different data sources, and cross-system data synchronization is achieved.

[0021] See Figure 3 As shown, the stream and batch processing engine module specifically includes: Metadata Alignment Unit: This unit unifies field names, types, and data granularity through the mapping relationship between heterogeneous data fields and unified audit fields, and adds metadata tags to data from different sources; Timestamp synchronization unit: The unit extracts the original timestamp of each data, aligns the data based on the original timestamp of the event, and controls the delay window of out-of-order data through Flink's Watermark mechanism; Event Stream Data Processing Unit: This unit converts events from different sources into audit events, aggregates and serializes cross-system events, and generates audit event stream data with global sequence numbers.

[0022] Specifically, fields from different sources are mapped and standardized to the format required by the audit system. Different data sources have different data types, and the data types and granularity are unified during mapping. Meta-information tags are added to each data entry based on its source and processing status. These tags include data source, data type, timestamp, and processing status. Regarding data granularity, it is ensured that data from different sources is standardized to the same granularity. For example, second-level timestamps are standardized to minute-level timestamps, or different log entries are aggregated according to the same time window. Extract the original timestamps from data from different sources, align all data according to the timestamps, and ensure that data from different sources can be compared and processed on the same time dimension. Use Flink's Watermark mechanism to handle out-of-order data, ensuring that delayed data can be processed within an appropriate time window. A Watermark is a logical timestamp that indicates the last valid time point in the data stream and controls the reception and processing of out-of-order data. The core purpose of the Watermark mechanism is to tolerate out-of-order data within a certain delay window. The raw events are converted into a unified audit event format. Audit events typically include event type, occurrence time, related fields, and additional metadata. For events from multiple systems, such as logs from multiple services or records from multiple databases, they are aggregated based on their correlation to form a global event set. A globally unique sequence number is added to all audit events to ensure the unique identification of events throughout the system. The aggregated event stream is then serialized and converted into a format suitable for storage or transmission.

[0023] See Figure 4 As shown, the intelligent desensitization and bloodline management module specifically includes: Sensitive Field Identification Unit: The unit identifies sensitive fields in audit data through a pre-trained NLP model and extracts sensitive information entities from the data through named entity recognition technology; Gradient desensitization unit: The unit reduces the accuracy of data based on the sensitivity of sensitive fields, and uses different desensitization techniques for each sensitive field, including encryption desensitization, masking desensitization, data generalization and data obfuscation; Data lineage marker: During the desensitization process, the unit adds a lineage marker to each piece of data, including the data source identifier, desensitization type, and data version number; Reversible mapping relationship chain unit: The unit is used to record the mapping relationship between desensitized data and original data and establish a mapping table, and restore the desensitized data to the original data through a reversible desensitization mechanism.

[0024] Specifically, NLP models are used to analyze audit data and identify potentially sensitive fields, such as ID card numbers, bank card numbers, phone numbers, and addresses. NLP models typically perform text analysis using pre-trained word vectors or Transformer architectures. NER technology is used to extract sensitive information entities from data. NER technology can identify sensitive data entities such as personal identification information, credit card numbers, and medical information. The method of data masking used depends on the sensitivity of the sensitive fields, including: Encryption and data masking involves encrypting data to prevent it from being read directly. Masking and desensitization partially mask sensitive fields, such as displaying a bank card number as "**** 1234"; Data generalization transforms specific numerical values ​​into broader ranges, such as transforming age 29 into "20-30 years old"; Data blurring involves blurring data to make it lose its accuracy, such as converting a date to "2025". Different levels of data anonymization strategies are applied to data with varying degrees of sensitivity. High-sensitivity data is subjected to strong encryption or complete masking, while low-sensitivity data may only be obfuscated. During data anonymization, a lineage marker is generated for each data entry, including: The data source identifier indicates which system or database the data comes from; What desensitization strategy (such as encryption, masking, generalization, obfuscation, etc.) was applied to the desensitization type record? The data version number identifies the processing version of the data, ensuring that a specific version can be traced after each data processing. Embedding lineage markers into the data, with each data entry containing a marker, allows for clear understanding of the processing history of each data entry during subsequent processing and tracing. During the de-identification process, a one-to-one mapping table is established to record the correspondence between de-identified data and original data. At the same time, a mapping table is constructed that contains the mappings of all sensitive fields before and after de-identification, which is convenient for subsequent data recovery.

[0025] See Figure 5 As shown, the tiered storage module specifically includes: Sensitivity Level Classification Unit: The unit classifies data into high-sensitivity data, medium-sensitivity data, low-sensitivity data, high-frequency data, medium-frequency data, and low-frequency data based on data sensitivity and data access frequency; Tiered storage unit: The unit classifies and stores data into an encrypted memory pool, an SSD hot storage area, and a cold backup cluster based on the characteristics of the partitioned data; Load balancing unit: The unit distributes data of different sensitivity levels through a consistent hashing algorithm, reduces data redistribution by increasing the number of virtual nodes, and avoids storage overload by periodically and dynamically adjusting the data hash distribution.

[0026] Specifically, sensitivity analysis is performed on the data, classifying it according to its content, nature, and degree of impact on privacy, including: Highly sensitive data: such as ID card numbers and bank account numbers; Sensitive data: such as name, contact information; Low-sensitivity data: such as publicly available information and non-sensitive business data; Based on access frequency, data is categorized into high-frequency, medium-frequency, and low-frequency data. Frequently accessed data should be preferentially stored on higher-performance storage devices. The data is further divided according to access frequency as follows: High-frequency data: data that is frequently accessed, such as real-time logs and hot data. Mid-frequency data: Data accessed only occasionally, such as historical business records; Low-frequency data: Data that is rarely accessed, such as archived data; Each data category determines its subsequent storage strategy and priority; Based on the sensitivity and access frequency of the data, data is allocated to different storage tiers, including encrypted memory pools, SSD hot storage areas, and cold backup clusters. Encrypted memory pools are suitable for highly sensitive data, providing fast access and data encryption; SSD hot storage areas are suitable for medium-sensitive data with high-frequency access, providing low latency and fast response; and cold backup clusters are suitable for data with low-frequency access, typically for long-term storage or backup data. Consistent hashing is used to ensure even data distribution. Data is allocated to different storage nodes based on its sensitivity level and frequency. Consistent hashing can avoid data migration and redistribution, improving storage scalability and reliability. By increasing the number of virtual nodes, the frequency of data redistribution is reduced, maintaining even data distribution and avoiding overload. The hash distribution of data is adjusted periodically according to storage load to avoid overloading certain storage nodes and ensure efficient utilization of storage resources.

[0027] See Figure 6 As shown, the multimodal safety monitoring module specifically includes: Stream processing layer unit: The unit includes a real-time metric deviation monitor based on a CUSUM control chart; Graph computation layer unit: The unit includes a dynamic temporal graph network modeler; Deep analysis layer unit: The unit includes a joint anomaly detection model that integrates a Transformer temporal encoder and a GNN graph neural network.

[0028] See Figure 6 As shown, the stream processing layer unit specifically includes: Key monitoring metrics are extracted from the audit event stream, including user access frequency, operation duration, session latency, system call frequency, number of network access packets, number of login failures, and frequency of sensitive operations. By calculating the mean and variance through historical statistical modeling, a behavioral baseline for each type of operator is established, while real-time indicators are standardized. The CUSUM algorithm continuously tracks indicator deviations, and when the accumulated deviations exceed a preset threshold, a deviation alarm is triggered.

[0029] Specifically, after acquiring the monitoring indicators, it is necessary to use historical data modeling to calculate the behavioral baseline of each type of operating entity. For each monitoring indicator, the mean and variance of historical data are calculated. The monitoring indicators collected in real time are standardized and compared with the historical baseline. The CUSUM algorithm is used to track and monitor deviations in metrics in real time. CUSUM helps to promptly detect deviations from the baseline when metric values ​​change. The basic principle of CUSUM is to calculate the deviation of the metric from the historical baseline at each time point and to identify anomalies by accumulating the deviation values. The formula is as follows: Positive CUSUM:

[0030] Reverse CUSUM:

[0031] in, The positive cumulative sum at the current moment. The cumulative sum in reverse order at the current moment. For standardized real-time metrics, To control the limits, the sensitivity to deviation is determined; When the cumulative value of either the forward or reverse CUSUM exceeds a preset threshold, the system considers that the indicator at that moment has deviated significantly and will trigger an alarm.

[0032] See Figure 6 As shown, the graph computation layer unit specifically includes: The entities in the audit data are mapped to nodes in the graph, and the operation behaviors are constructed as edges with timestamps, forming a dynamic time sequence graph structure, with each edge carrying behavior attributes; Extract the interaction frequency, path depth, and behavioral pattern graph structure features of nodes to identify abnormal relationship patterns; Track the evolution trend of the graph over time, detect abrupt changes, update the graph structure in real time, and obtain the current system status.

[0033] See Figure 6 As shown, the deep analysis layer unit specifically includes: Data from the stream processing layer and the graph computation layer is fused to obtain time series data, and the Transformer encoder is used to capture long-term dependencies and temporal patterns in the time series data. A joint model is trained by using self-supervised learning and contrastive learning to map normal and abnormal behaviors into distinguishable feature representations. Input the real-time behavioral sequence and graph structure of the audit data into the model, and output the risk score and anomaly type label for each behavioral entity.

[0034] See Figure 7 As shown, the tiered response and blockchain evidence storage module specifically includes: Tiered Response Unit: This unit defines corresponding levels, including mild, moderate, and high responses, based on real-time response tiers according to event classifications of various audit data. Blockchain Evidence Storage Unit: The unit stores key audit data, abnormal events, and response results in the blockchain, generates a fixed-length hash value for the event data through a hash algorithm, and performs automated evidence storage through a smart contract.

[0035] Specifically, based on different types of audit data, the first step is to classify events. Event classification criteria may include outliers in the audit data, violations, and system failures. Classification standards may involve multiple factors, such as the frequency of the event, the scope of its impact, and the degree of risk. A risk score for each event is then established, calculated using the following formula:

[0036] in, To score risk, , The various factors that influence the score, , The weights of the corresponding factors; Based on risk scoring Define the response level: Mild response: when When the response is below the mild threshold, the event is classified as a mild response, such as a simple warning or reminder to the user. Moderate response: When the mild threshold is ≤ When the threshold is less than medium, the event is classified as a medium response, requiring further investigation and human intervention. High responsiveness: when When the threshold is ≥ medium, the event is classified as high-response, such as the system automatically performing repairs, issuing alarms, and possibly initiating emergency handling procedures.

[0037] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, the above description focuses on specific embodiments of this specification. Additionally, the processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired results. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0038] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0039] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A big data-based audit data security management system, characterized in that, include: Distributed crawler collection module: The module is deployed on the nodes of the audit data source and collects structured audit logs, unstructured operation records and raw network traffic packets in real time. After compression and encryption through the data channel, it is transmitted to the stream and batch processing engine module. Integrated batch and stream processing engine module: This module performs metadata alignment and timestamp synchronization on heterogeneous data to generate audit event stream data with unified spatiotemporal dimensions; Intelligent desensitization and lineage management module: The module identifies sensitive fields in the audit data, performs gradient desensitization on the audit data, and builds a reversible mapping relationship chain between the desensitized data and the original data by adding data lineage tags; Tiered storage module: Based on data sensitivity level and access frequency, the module stores data in layers in an encrypted memory pool, an SSD hot storage area, and a cold backup cluster, and performs load balancing processing on data shards through a consistent hashing algorithm; Multimodal security monitoring module: The module monitors audit data in real time by constructing a parallel three-level processing pipeline, including a stream processing layer, a graph computing layer, and a deep analysis layer; Tiered response and blockchain evidence storage module: The module triggers a multi-level response mechanism based on the data risk level, including real-time session blocking, dynamic permission downgrading, and audit mark tracing, and writes the hash value of key audit events into the permissioned chain to build an tamper-proof evidence chain.

2. The big data-based audit data security management system according to claim 1, characterized in that, The distributed crawler acquisition module specifically includes: Structured data unit: The unit parses the real-time log stream by event type and batch maps and parses historical data by field; Unstructured data unit: The unit unifies and slices unstructured data, and performs semantic parsing and behavioral intent recognition through OCR text recognition; Network traffic unit: The unit is used to extract key fields such as IP, port, URL, and file transfer content, and to annotate the data packet time series and transmission path; Cross-system data pool unit: The unit stores data of different formats from different systems through a distributed database, and constructs a cross-system audit data pool for data classification and data synchronization; Compression and Encryption Unit: The unit compresses the data using a compression algorithm and encrypts the data using an encryption algorithm, then transmits the processed data to the batch processing engine module.

3. The big data-based audit data security management system according to claim 1, characterized in that, The integrated stream and batch processing engine module specifically includes: Metadata Alignment Unit: This unit unifies field names, types, and data granularity through the mapping relationship between heterogeneous data fields and unified audit fields, and adds metadata tags to data from different sources; Timestamp synchronization unit: The unit extracts the original timestamp of each data, aligns the data based on the original timestamp of the event, and controls the delay window of out-of-order data through Flink's Watermark mechanism; Event Stream Data Processing Unit: This unit converts events from different sources into audit events, aggregates and serializes cross-system events, and generates audit event stream data with global sequence numbers.

4. The big data-based audit data security management system according to claim 1, characterized in that, The intelligent desensitization and bloodline management module specifically includes: Sensitive Field Identification Unit: The unit identifies sensitive fields in audit data through a pre-trained NLP model and extracts sensitive information entities from the data through named entity recognition technology; Gradient desensitization unit: The unit reduces the accuracy of data based on the sensitivity of sensitive fields, and uses different desensitization techniques for each sensitive field, including encryption desensitization, masking desensitization, data generalization and data obfuscation; Data lineage marker: During the desensitization process, the unit adds a lineage marker to each piece of data, including the data source identifier, desensitization type, and data version number; Reversible mapping relationship chain unit: The unit is used to record the mapping relationship between desensitized data and original data and establish a mapping table, and restore the desensitized data to the original data through a reversible desensitization mechanism.

5. The big data-based audit data security management system according to claim 1, characterized in that, The hierarchical storage module specifically includes: Sensitivity Level Classification Unit: The unit classifies data into high-sensitivity data, medium-sensitivity data, low-sensitivity data, high-frequency data, medium-frequency data, and low-frequency data based on data sensitivity and data access frequency; Tiered storage unit: The unit classifies and stores data into an encrypted memory pool, an SSD hot storage area, and a cold backup cluster based on the characteristics of the partitioned data; Load balancing unit: The unit distributes data of different sensitivity levels through a consistent hashing algorithm, reduces data redistribution by increasing the number of virtual nodes, and avoids storage overload by periodically and dynamically adjusting the data hash distribution.

6. The big data-based audit data security management system according to claim 1, characterized in that, The multimodal safety monitoring module specifically includes: Stream processing layer unit: The unit includes a real-time metric deviation monitor based on a CUSUM control chart; Graph computation layer unit: The unit includes a dynamic temporal graph network modeler; Deep analysis layer unit: The unit includes a joint anomaly detection model that integrates a Transformer temporal encoder and a GNN graph neural network.

7. The big data-based audit data security management system according to claim 6, characterized in that, The stream processing layer unit specifically includes: Key monitoring metrics are extracted from the audit event stream, including user access frequency, operation duration, session latency, system call frequency, number of network access packets, number of login failures, and frequency of sensitive operations. By calculating the mean and variance through historical statistical modeling, a behavioral baseline for each type of operator is established, while real-time indicators are standardized. The CUSUM algorithm continuously tracks indicator deviations, and when the accumulated deviations exceed a preset threshold, a deviation alarm is triggered.

8. The big data-based audit data security management system according to claim 6, characterized in that, The graph computation layer unit specifically includes: The entities in the audit data are mapped to nodes in the graph, and the operation behaviors are constructed as edges with timestamps, forming a dynamic time sequence graph structure, with each edge carrying behavior attributes; Extract the interaction frequency, path depth, and behavioral pattern graph structure features of nodes to identify abnormal relationship patterns; Track the evolution trend of the graph over time, detect abrupt changes, update the graph structure in real time, and obtain the current system status.

9. The big data-based audit data security management system according to claim 6, characterized in that, The deep analysis layer unit specifically includes: Data from the stream processing layer and the graph computation layer is fused to obtain time series data, and the Transformer encoder is used to capture long-term dependencies and temporal patterns in the time series data. A joint model is trained by using self-supervised learning and contrastive learning to map normal and abnormal behaviors into distinguishable feature representations. Input the real-time behavioral sequence and graph structure of the audit data into the model, and output the risk score and anomaly type label for each behavioral entity.

10. The big data-based audit data security management system according to claim 1, characterized in that, The hierarchical response and blockchain evidence storage module specifically includes: Tiered Response Unit: This unit defines corresponding levels, including mild, moderate, and high responses, based on real-time response tiers according to event classifications of various audit data. Blockchain Evidence Storage Unit: The unit stores key audit data, abnormal events, and response results in the blockchain, generates a fixed-length hash value for the event data through a hash algorithm, and performs automated evidence storage through a smart contract.

Images