Real-time analysis system based on business data isolation and low-intrusion data synchronization method

Abstract

This invention discloses a real-time analysis system and a low-intrusion data synchronization method based on business data isolation, belonging to the field of big data analysis and data security technology. It includes: a business data layer for storing raw business data, comprising a business database cluster and a business system; a synchronization proxy module deployed between the business data layer and the analysis data layer, which performs non-intrusive data capture by parsing incremental database logs and performs field-level encryption and fragmented transmission of the data; and an analysis data layer that uses a columnar storage architecture to store anonymized analysis data, and achieves real-time querying and access control of encrypted data through a unified query gateway and a dynamic anonymization module. This invention enables secure, efficient, and real-time data isolation analysis through an innovative combination of technologies.

Description

Technical Field

[0001] This invention relates to the fields of big data analytics and data security, specifically to a real-time analytics system based on business data isolation and a low-intrusion data synchronization method. Background Technology

[0002] In today's digital age, data has become a vital asset for businesses and organizations, and data analytics is crucial for decision-making and development. However, existing data analytics solutions face numerous unresolved issues in practical applications, as follows:

[0003] (1) Data security risks are prominent

[0004] The risk of sensitive data leakage is significant: In traditional data analytics solutions, the analysis system often directly accesses the production database, a method fraught with serious security vulnerabilities. For example, in the financial industry, customer transaction data, account information, and personal identification information are all highly sensitive data. Once the production database is illegally accessed or internal personnel mishandle it, sensitive data can easily be leaked. This can not only cause huge economic losses and privacy violations for customers, but also expose financial institutions to legal risks, reputational crises, and even affect the stability of the entire financial system.

[0005] Compliance requirements are difficult to meet: With the promulgation and implementation of laws and regulations such as the Data Security Law and the Personal Information Protection Law, higher requirements have been placed on the security and compliance of data processing. Traditional solutions lack effective physical isolation and dynamic de-identification mechanisms for data, making it difficult to meet the explicit provisions of regulations on the protection of sensitive data, resulting in compliance risks for enterprises when conducting data analysis business.

[0006] (2) Serious performance interference problems

[0007] Business system efficiency is impacted: Analytical queries typically require complex calculations and processing of large amounts of data, which consumes significant business database resources such as CPU, memory, and disk I / O. In busy systems, the concurrent execution of numerous analytical queries can significantly slow down the response speed of the business database, or even cause congestion, resulting in a substantial decrease in the efficiency of normal business transaction processing and severely impacting the company's business operations.

[0008] System stability is threatened: Long-term high-load analysis and querying may cause business database failures, such as server crashes and data loss, which in turn affect the stability and availability of the entire business system and cause incalculable losses to the enterprise.

[0009] (3) Data timeliness cannot meet the requirements.

[0010] Traditional ETL methods are outdated: Traditional ETL (Extract-Transform-Load) solutions typically use daily batch synchronization for data processing, which cannot reflect the latest changes in business data in real time. In today's rapidly changing market environment, enterprises need to acquire and analyze data promptly to make accurate decisions. For example, in e-commerce, real-time analysis of user purchasing behavior and market trends is crucial for timely adjustments to marketing strategies, but traditional daily batch synchronization methods result in significant delays in analysis results, failing to meet the real-time decision-making needs of enterprises.

[0011] Limited by real-time business scenarios: With the continuous emergence of real-time business scenarios such as the Internet of Things, real-time marketing, and financial risk control, the demand for real-time data analysis is growing. Traditional solutions, due to poor data timeliness, cannot play an effective role in these scenarios, thus limiting the expansion and innovation of enterprise businesses.

[0012] (4) Limitations of existing technical solutions

[0013] The patented technology has shortcomings: For example, patent CN202411383639.2 discloses a data anonymization method, which mainly focuses on the data anonymization technology itself, but does not address the performance impact of real-time synchronization and analysis systems on business systems. In practical applications, the real-time nature of data synchronization and the performance interference of analysis systems on business systems are two critical issues that urgently need to be addressed, and this patent fails to address them, resulting in certain limitations in its practical application.

[0014] Intrusive synchronization methods have significant drawbacks: Traditional data synchronization technologies often employ intrusive methods based on API interfaces or requiring modifications to business systems. This approach necessitates code modifications to business systems, increasing development and maintenance costs, potentially introducing new system vulnerabilities, and significantly impacting the normal operation of business systems. Consequently, it fails to meet enterprises' needs for low-intrusive data synchronization.

[0015] Static data masking solutions have insufficient security: Traditional static data masking methods perform data masking during the data storage stage. While this approach protects data privacy to some extent, sensitive data still faces the risk of persistent exposure at the storage layer. Once the data at the storage layer is leaked, the masked data may still contain valuable information, posing a threat to data security.

[0016] In summary, existing data analysis solutions have significant shortcomings in terms of data security, performance degradation, and data timeliness, failing to meet the demands of enterprises for "low-intrusion, high-timeliness, and strong security" data analysis in today's highly sensitive data scenarios. A new technical solution is urgently needed to address these issues. Summary of the Invention

[0017] The technical objective of this invention is to address the above-mentioned shortcomings by providing a real-time analysis system based on business data isolation and a low-intrusion data synchronization method, which can achieve secure, efficient, and real-time data isolation analysis through an innovative combination of technologies.

[0018] The technical solution adopted by this invention to solve its technical problem is:

[0019] Real-time analytics systems based on business data isolation include:

[0020] The business data layer is used to store raw business data, including the business database cluster and business systems;

[0021] The synchronous proxy module is deployed between the business data layer and the analysis data layer. It achieves non-intrusive data capture by parsing the incremental logs of the database and performs field-level encryption and fragmented transmission on the data.

[0022] The analysis data layer uses a columnar storage architecture to store the anonymized analysis data, and a unified query gateway and dynamic anonymization module enable real-time querying and access control of encrypted data.

[0023] Furthermore, the synchronization agent module achieves incremental data capture by parsing MySQL binlog, PostgreSQL redolog, or Oracle archive logs, without requiring modification to the business system code.

[0024] Furthermore, the business data layer and the analysis data layer are isolated by a one-way firewall, allowing only the synchronization proxy module to transmit data to the analysis data layer through SSL / TLS encrypted ports (443 / 9443), and prohibiting reverse communication;

[0025] The synchronous proxy module has a built-in data verification mechanism. When data is encrypted and fragmented, a hash check code (such as SHA-256) is generated. After analyzing the data received by the data layer, hash comparison and timestamp verification are performed to ensure that there is no loss or tampering during data transmission. For abnormal data (such as verification failure), the snapshot completion channel is automatically triggered to repair the data.

[0026] Furthermore, the analysis data layer adopts Parquet, ORC, or Delta Lake columnar storage format, and accelerates the scanning and aggregation of encrypted data through column-level indexes (prefix indexes / bitmap indexes).

[0027] Furthermore, the synchronization proxy module includes a dual-channel synchronization mechanism:

[0028] Real-time streaming channel, processing incremental data in seconds, with latency controlled within 500ms;

[0029] The snapshot completion channel enables data repair after a failure by using full data snapshots and hash verification.

[0030] When the snapshot completion channel detects a data verification failure (hash value mismatch / timestamp abnormality), it automatically triggers full synchronization and transmits only the missing data fragments through difference comparison.

[0031] Furthermore, the synchronization agent module supports field-level dynamic encryption using AES-256, RSA-2048, or the national cryptographic algorithm SM4, and the encryption strategy can be configured according to the data sensitivity level (top secret / confidential / normal).

[0032] The fragmented transmission mechanism splits encrypted data into fragments of 128KB-1MB based on data type (structured / unstructured), timestamp (fragmented by hour), or hash value (modulo user ID), and transmits them in parallel through multiple channels.

[0033] Furthermore, the dynamic de-identification module of the analysis data layer supports a three-level permission policy:

[0034] Administrator privileges: View all data (including sensitive fields);

[0035] Analyst permissions: View partial data according to masking rules (e.g., the middle 8 digits of an ID number are displayed as "****");

[0036] Researcher permissions: Only view anonymized statistical data (such as age range / disease classification);

[0037] The dynamic desensitization engine configures field-level desensitization rules based on user roles (administrators, analysts, researchers), including masking (e.g., displaying the middle 8 digits of an ID number as "****"), generalization (e.g., displaying age by range and hiding the specific value), and deletion (directly blocking highly sensitive fields, such as bank card CVV codes).

[0038] The desensitization rules are bound to the query statement in real time and applied dynamically when the query is executed by the distributed computing engine (Spark / Flink), avoiding the persistent exposure of sensitive data at the storage layer.

[0039] Furthermore, the unified query gateway supports the conversion from standard SQL to encrypted columnar queries, and integrates predicate pushdown (pushing WHERE conditions to the storage layer) and vectorized execution (batch processing of data columns) optimizations.

[0040] Furthermore, the synchronization agent module supports multi-source data access, including relational databases (MySQL / PostgreSQL / Oracle), non-relational databases (MongoDB / Cassandra), and real-time streaming data (Kafka / Pulsar);

[0041] The analytical data layer uses a distributed computing framework (Spark / Flink) to enable elastic scaling of query tasks, supporting the dynamic addition of computing nodes to cope with petabyte-scale data growth.

[0042] This invention also claims protection for a low-intrusion data synchronization method based on business data isolation, which is based on the above-described system implementation and includes the following steps:

[0043] (1) The synchronous agent module listens to the business database logs in real time and captures INSERT / UPDATE / DELETE incremental data change events;

[0044] (2) Perform field-level dynamic encryption (AES-256 / SM4) on incremental data and shard it according to data type / timestamp / hash value;

[0045] (3) The fragmented data is transmitted to the analysis data layer via a one-way encrypted channel (SSL / TLS), with an accompanying SHA-256 checksum;

[0046] (4) The data layer analyzes the received data and, if it passes the verification, decrypts and stores it in columnar storage format;

[0047] (5) Users send SQL requests through the unified query gateway. The dynamic desensitization module performs field-level desensitization according to the user's role and then returns the results.

[0048] Compared with existing technologies, the real-time analysis system and low-intrusion data synchronization method based on business data isolation of the present invention have the following advantages:

[0049] This invention achieves breakthrough improvements in data security, system performance, and business adaptability through the deep integration of a three-layer physical isolation architecture and a low-intrusion technology system, as detailed below:

[0050] 1. Breakthrough in data security and compliance.

[0051] Building a full lifecycle security barrier: Through a three-tiered protection system of physical isolation architecture, field-level dynamic encryption, and real-time data masking during the query phase, sensitive data is made "usable but not visible." In financial transaction scenarios, customer bank card numbers, transaction amounts, and other data are encrypted using AES-256 during transmission, stored in ciphertext form, and dynamically masked during queries based on analyst permissions (e.g., only displaying the last four digits), thus blocking the risk of data leakage at the source. This solution improves security by more than 300% compared to traditional static data masking solutions and fully meets the compliance requirements of the Data Security Law and the Personal Information Protection Law for sensitive data processing.

[0052] Meeting the specific needs of highly sensitive fields: In medical research scenarios, the system supports fine-grained anonymization control of medical record data: researchers can only access anonymized diagnostic results (such as "respiratory system disease" rather than specific disease names) and statistical data; patient names, ID numbers, and other information are completely masked. This "data usable but privacy not leaked" mechanism supports big data analysis in medical research while protecting patient privacy, complying with industry standards.

[0053] 2. Reduce the load on business systems.

[0054] By adopting non-intrusive log parsing technology, the business database does not need to provide additional resources for analysis and querying. The CPU utilization rate drops from 60%-80% in traditional solutions to below 10%, and the I / O throughput is greatly improved, completely solving the problem of resource contention between analysis and querying and business transactions.

[0055] 3. Data consistency guarantee.

[0056] The snapshot completion channel automatically triggers full data synchronization during system initialization or network failures. Through hash verification and timestamp comparison mechanisms, it accurately identifies missing data and completes the completion. This ensures real-time synchronization between analytical data and business data, avoiding decision-making errors caused by data inconsistencies.

[0057] 4. Advantages in business adaptability and scalability.

[0058] Multi-source heterogeneous data fusion capability: The synchronization agent module supports access from multiple sources, including relational databases (such as Oracle / MySQL), non-relational databases (such as MongoDB), and real-time streaming data (such as Kafka), and achieves standardized processing through a unified data transformation engine.

[0059] Flexible hierarchical access control: The dynamic data masking module supports field-level access control based on user roles. For example, bank administrators can view all transaction data (including sensitive fields), risk analysts can only view the masked transaction amount (e.g., "*5678") and transaction time, and research personnel can only access anonymized transaction statistics reports. This fine-grained access control mechanism meets the differentiated data usage needs of different departments and positions within financial institutions, while reducing the risk of data misuse.

[0060] This invention achieves the data analysis goals of "safety and compliance, high efficiency and low power consumption, and real-time intelligence" through technological innovation, providing a solution with both technological advancement and commercial feasibility for highly sensitive data scenarios, and filling the gap in existing technologies in the fields of physical isolation analysis and low-intrusion synchronization. Attached Figure Description

[0061] Figure 1This is a system architecture diagram of a real-time analysis system based on business data isolation provided in one embodiment of the present invention;

[0062] Figure 2 This is a flowchart illustrating the incremental data capture and encryption process provided in one embodiment of the present invention;

[0063] Figure 3 This is a timing diagram illustrating dynamic desensitization and query execution provided in one embodiment of the present invention;

[0064] Figure 4 This is a schematic diagram of a dual-channel data synchronization mechanism provided in an embodiment of the present invention. Detailed Implementation

[0065] The present invention will be further described below with reference to specific embodiments.

[0066] This invention provides a real-time analysis system based on business data isolation. The system adopts a three-layer architecture design: a business data layer, a synchronization proxy module, and an analysis data layer. Each layer is connected through a unidirectional encrypted data stream, forming a physical isolation barrier.

[0067] Business data layer: Deploys business database clusters (such as MySQL / PostgreSQL) with business systems, stores raw business data, and is the data source.

[0068] Synchronization Proxy Module: Deployed between the business data layer and the analytics data layer, isolated by a one-way firewall (SSL / TLS encrypted ports), allowing data to flow only from the business layer to the analytics layer. The synchronization proxy module achieves non-intrusive data capture by parsing incremental database logs and performs field-level encryption and fragmented transmission of the data.

[0069] The analytical data layer employs a columnar storage cluster (Parquet / ORC format) and a distributed computing framework (Spark / Flink) to support the storage and real-time analysis of encrypted data. This layer stores anonymized analytical data and enables real-time querying and access control of the encrypted data through a unified query gateway and a dynamic anonymization module.

[0070] This system implements a one-way data flow security mechanism.

[0071] Data can only flow from the business data layer to the analysis data layer via the synchronization proxy module; reverse transmission is prohibited. The business data layer and the analysis data layer are isolated by a one-way firewall, allowing the synchronization proxy module to transmit data to the analysis data layer only through SSL / TLS encrypted ports (443 / 9443), prohibiting reverse communication. The synchronization proxy module achieves non-intrusive data capture through database log parsing, preventing the analysis system from directly accessing the business database and eliminating the risk of data leakage at the architectural level.

[0072] The synchronization proxy module includes the following technologies:

[0073] (1) Non-intrusive incremental data capture:

[0074] It employs database log parsing technologies (such as MySQL binlog, PostgreSQL redo log, or Oracle) to monitor incremental changes in the business database in real time without modifying the business system code. The log parsing engine supports parallel parsing across multiple database clusters, ensuring the real-time performance and integrity of data capture.

[0075] (2) Field-level dynamic encryption and fragmented transmission:

[0076] The encryption module supports field-level dynamic encryption using AES-256, RSA-2048, or the national cryptographic algorithm SM4. Encryption strategies can be configured according to data sensitivity levels (top secret / confidential / normal), and different encryption strategies can be configured based on data sensitivity levels (such as ID card number, bank card number) to achieve field-level dynamic encryption.

[0077] The fragmented transmission manager fragments encrypted data based on data type (structured / unstructured), timestamp (fragmented by hour / day), or hash value (such as user ID hash modulo), splitting the encrypted data into fragments of 128KB-1MB. Through multi-channel parallel transmission, it improves throughput while reducing the transmission pressure of a single channel.

[0078] (3) Dual-channel synchronization mechanism:

[0079] Real-time streaming channel: processes incremental data at the second level, suitable for continuous data synchronization in normal business scenarios, with latency controlled within 500ms.

[0080] Snapshot Completion Channel: Used for full synchronization during system initialization, network failures, or data consistency checks. It automatically identifies missing data and triggers completion through data comparison and verification (hash value + timestamp), ensuring the integrity of the analyzed data. It ensures the integrity of transmitted data by comparing SHA-256 hash checksums with timestamps; automatic retransmission or snapshot completion is triggered when verification fails. The snapshot completion channel achieves data repair after failures through full data snapshots and hash verification. When data verification failure is detected (hash value mismatch / timestamp anomaly), it automatically triggers full synchronization and transmits only missing data fragments through difference comparison.

[0081] The analytical data layer includes:

[0082] (1) Columnar storage and query optimization:

[0083] Employing columnar storage formats such as Parquet, ORC, or Delta Lake allows for contiguous storage of similar fields, reducing I / O reads and improving scan efficiency. For encrypted data, column-level indexes (such as prefix indexes and bitmap indexes) accelerate the scanning and aggregation of encrypted data, thus speeding up queries.

[0084] (2) The unified query gateway supports the conversion from standard SQL to encrypted columnar queries and integrates predicate pushdown (pushing filtering conditions to the storage layer) and vectorized execution (batch processing of data columns) optimization strategies.

[0085] (3) Dynamic desensitization and access control:

[0086] The dynamic data masking module in the analysis data layer supports a three-level permission policy:

[0087] Administrator privileges: View all data (including sensitive fields);

[0088] Analyst permissions: View partial data according to masking rules (e.g., the middle 8 digits of an ID number are displayed as "****");

[0089] Researcher permissions: Only view anonymized statistical data (such as age range / disease classification).

[0090] The dynamic data masking engine configures field-level data masking rules based on user roles (administrators, analysts, researchers):

[0091] Masking strategy: For example, display the middle 8 digits of the ID number as "****";

[0092] Generalization strategy: For example, display age ranges (20-30 years old) and hide specific values;

[0093] Deletion strategy: Highly sensitive fields (such as bank card CVV codes) are directly blocked from display.

[0094] The data masking logic is bound to the query statement in real time, and data masking is performed only as needed during the query execution phase. It is dynamically applied when the distributed computing engine (Spark / Flink) executes the query, thus avoiding the persistent exposure of sensitive data in the storage layer.

[0095] The system employs the following solutions to ensure data security and integrity:

[0096] Data transmission security:

[0097] The business data layer and the analysis data layer are isolated by a one-way firewall, with only SSL / TLS encrypted ports (such as port 443) open. Data transmission is encrypted using encryption algorithms to prevent data eavesdropping and tampering.

[0098] Data integrity verification mechanism:

[0099] The synchronization proxy module generates hash checksums (such as SHA-256) during data encryption and fragmentation. After analyzing the received data from the data layer, it performs hash comparison and timestamp verification to ensure that no data is lost or tampered with during transmission. For abnormal data (such as verification failures), the snapshot completion channel is automatically triggered to repair the data.

[0100] This system enables multi-source data access.

[0101] Heterogeneous data source adaptation: The synchronization agent module supports access from multiple sources, including relational databases (MySQL / PostgreSQL / Oracle), non-relational databases (MongoDB / Cassandra), and real-time streaming data (Kafka / Pulsar). Through the unified log parsing and data transformation module, heterogeneous data is standardized into a unified format (such as JSON / Parquet), enabling multi-source data fusion analysis.

[0102] The analytical data layer uses a distributed computing framework (Spark / Flink) to enable elastic scaling of query tasks, supporting the dynamic addition of computing nodes to cope with petabyte-scale data growth.

[0103] like Figure 1 The diagram shown illustrates the system architecture, demonstrating the physical isolation between the business layer, synchronization proxy, and analysis layer. The business data layer, synchronization proxy module, and analysis data layer are connected via a one-way encrypted data stream, clearly illustrating the physical isolation between each layer and the direction of data transmission.

[0104] Figure 2 The process of incremental data capture and encryption is demonstrated, including step ①: monitoring database logs and monitoring changes in the database logs of the business data layer in real time; step ②: field-level encryption, performing field-level encryption on the captured incremental data; and step ③: fragmented transmission, transmitting the encrypted data in fragments to the analysis data layer.

[0105] Figure 3 The sequence diagram for dynamic masking and query execution illustrates the workflow and time sequence of the dynamic masking module during the query process, as well as the transformation and execution process of the query statement by the unified query gateway.

[0106] Figure 4 The diagram illustrates a dual-channel data synchronization mechanism, including two synchronization methods: real-time streaming and snapshot completion. Real-time streaming handles incremental data generated in real time, while snapshot completion performs full data synchronization during system initialization or data anomalies, ensuring data integrity and consistency.

[0107] This system enables low-intrusion data synchronization, and the process is as follows:

[0108] Incremental data capture: The synchronization agent module monitors the business database logs in real time and captures incremental data records when data changes (INSERT / UPDATE / DELETE) are detected;

[0109] Field-level encryption: Based on preset sensitive field rules, the captured incremental data is encrypted using AES-256, RSA, or SM4 algorithms to ensure that sensitive information is desensitized before transmission;

[0110] Fragmentation and Verification Generation: Fragment the encrypted data according to data type, timestamp, or hash value, and generate the hash verification code and timestamp for each fragment;

[0111] One-way encrypted transmission: Fragmented data is sent to the analysis data layer via an SSL / TLS encrypted channel. The transmission process is controlled by a one-way firewall, which prohibits reverse communication.

[0112] Data verification and storage: The data analysis layer performs hash and timestamp verification on the received data. After verification, the data is decrypted and written to the storage cluster in columnar storage format (Parquet / ORC).

[0113] Real-time query and dynamic data masking: Users send SQL query requests through a unified query gateway. The system parses the SQL and generates an optimized query plan. When reading encrypted data, the dynamic data masking module is triggered. Field-level data masking is performed according to user permissions, and the results are returned.

[0114] Through the above architecture and process design, the core technical goals of "low-intrusive data synchronization, physical isolation security analysis, real-time dynamic desensitization, and efficient query optimization" have been achieved, which is suitable for compliance analysis needs in highly sensitive data scenarios such as finance, healthcare, and government.

[0115] This invention also provides a low-intrusion data synchronization method based on business data isolation. This method is implemented based on the low-intrusion data synchronization system based on business data isolation described in the above embodiments, and includes the following steps:

[0116] (1) The synchronous agent module listens to the business database logs in real time and captures INSERT / UPDATE / DELETE incremental data change events;

[0117] (2) Perform field-level dynamic encryption (AES-256 / SM4) on incremental data and shard it according to data type / timestamp / hash value;

[0118] (3) The fragmented data is transmitted to the analysis data layer via a one-way encrypted channel (SSL / TLS), with an accompanying SHA-256 checksum;

[0119] (4) The data layer analyzes the received data and, if it passes the verification, decrypts and stores it in columnar storage format;

[0120] (5) Users send SQL requests through the unified query gateway. The dynamic desensitization module performs field-level desensitization according to the user's role and then returns the results.

[0121] Through the specific embodiments described above, those skilled in the art can easily implement the present invention. However, it should be understood that the present invention is not limited to the specific embodiments described above. Based on the disclosed embodiments, those skilled in the art can arbitrarily combine different technical features to achieve different technical solutions.

[0122] Except for the technical features described in the specification, all other technologies are known to those skilled in the art.

Claims

1. A real-time analysis system based on business data isolation, characterized in that, include: The business data layer is used to store raw business data, including the business database cluster and business systems; The synchronous proxy module is deployed between the business data layer and the analysis data layer. It achieves non-intrusive data capture by parsing incremental database logs and performs field-level encryption and fragmented transmission on the data. The analysis data layer uses a columnar storage architecture to store the anonymized analysis data, and a unified query gateway and dynamic anonymization module enable real-time querying and access control of encrypted data. The synchronization agent module captures incremental data by parsing MySQL binlog, PostgreSQL redo log, or Oracle archive log, without requiring modification to the business system code. The business data layer and the analysis data layer are isolated by a one-way firewall, allowing only the synchronization proxy module to transmit data to the analysis data layer through an SSL / TLS encrypted port, and prohibiting reverse communication; The synchronous proxy module has a built-in data verification mechanism that generates a hash check code when the data is encrypted and fragmented. After analyzing the data received by the data layer, it performs hash comparison and timestamp verification to ensure that there is no loss or tampering during data transmission. For abnormal data, it automatically triggers the snapshot completion channel to repair the data. The synchronization proxy module includes a dual-channel synchronization mechanism: Real-time streaming channel, processing incremental data in seconds, with latency controlled within 500ms; The snapshot completion channel enables data repair after a failure by using full data snapshots and hash verification. When the snapshot completion channel detects a data verification failure, it automatically triggers full synchronization and transmits only the missing data fragments through difference comparison. The synchronization proxy module supports field-level dynamic encryption using AES-256, RSA-2048, or the national cryptographic algorithm SM4, with encryption strategies configured according to data sensitivity levels. The fragmented transmission mechanism splits encrypted data into fragments ranging from 128KB to 1MB based on data type, timestamp, or hash value, and transmits them in parallel through multiple channels.

2. The real-time analysis system based on business data isolation according to claim 1, characterized in that, The analytical data layer uses Parquet, ORC, or Delta Lake columnar storage formats, and accelerates the scanning and aggregation of encrypted data through column-level indexes.

3. The real-time analysis system based on business data isolation according to claim 1, characterized in that, The dynamic data masking module in the analysis data layer supports a three-level permission policy: Administrator privileges: View all data; Analyst permissions: View partial data according to mask rules; Researchers Permissions: Only view anonymized statistics; The dynamic masking engine configures field-level masking rules based on user roles, including masking, generalization, and deletion. The de-identification rules are bound to the query statement in real time and applied dynamically when the distributed computing engine executes the query.

4. The real-time analysis system based on business data isolation according to claim 1 or 3, characterized in that, The unified query gateway supports the conversion from standard SQL to encrypted columnar queries and integrates predicate pushdown and vectorized execution optimization.

5. The real-time analysis system based on business data isolation according to claim 1, characterized in that, The synchronization agent module supports multi-source data access, including relational databases, non-relational databases, and real-time streaming data; The analytical data layer uses a distributed computing framework to enable elastic scaling of query tasks, supporting the dynamic addition of computing nodes to cope with petabyte-level data growth.

6. A low-intrusion data synchronization method based on business data isolation, characterized in that: This method, implemented based on the system described in any one of claims 1 to 5, includes the following steps: (1) The synchronous agent module monitors the business database logs in real time and captures INSERT / UPDATE / DELETE incremental data change events; (2) Perform field-level dynamic encryption on incremental data and shard it according to data type / timestamp / hash value; (3) The fragmented data is transmitted to the analysis data layer through a one-way encrypted channel, with an SHA-256 checksum attached; (4) The data analysis layer verifies the received data, and after passing the verification, it decrypts and stores the data in a columnar storage format; (5) Users send SQL requests through the unified query gateway. The dynamic desensitization module performs field-level desensitization according to the user's role and then returns the results.

Images