A data security isolation method and system based on access permission verification
By receiving data access requests, parsing tenant information and data scope, obtaining access permission version information of the target data block, identifying differences and generating correction information, and updating the row-level access permission bitmap in place, the problem of data security isolation failure in multi-tenant cloud data warehouses is solved, achieving efficient data security isolation and sensitive data protection.
Patent Information
- Application Number
- CN202611524855.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-09-29
- Publication Date
- 2026-11-17
AI Technical Summary
In multi-tenant cloud data warehouses, the lack of intermediate result caching and reuse and security rule version verification leads to data security isolation failure and sensitive data leakage.
By receiving data access requests, parsing tenant identification information and data range, obtaining access permission version information of the target data block, identifying differences, generating row-level access permission correction information, updating the row-level access permission bitmap in place, and performing permission verification to ensure data security isolation.
It enables dynamic and efficient management of data security isolation in multi-tenant distributed computing platforms, avoids sensitive data leakage, and improves system response speed and data processing efficiency.
Smart Images

Figure CN123116307A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of data security isolation technology, and more specifically, to a data security isolation method and system based on access permission verification. Background Technology
[0002] In a multi-tenant cloud data warehouse, upper-level tenants access lower-level data across tenants through logical views with mounted Row-Level Security (RLS) rules. The query optimizer is responsible for pushing RLS predicates down to the storage scan layer and caching intermediate results of frequently occurring join subqueries in memory to improve reuse efficiency. This mechanism works normally when the rules are stable. However, after compliance administrators modify RLS rules online, metadata is refreshed, and cache invalidation signals are issued, the headquarters statistical tasks still executing in the pipeline enter a competition window between rule versions and execution status. The problems are concentrated on three levels: First, the optimizer only uses the query structure hash as the retrieval key for the intermediate result cache and does not establish a strong check with the RLS policy version at the time of generation. Therefore, after the metadata invalidation signal arrives, it still hits and reuses the cache block generated by the old rule. Secondly, the vectorized execution engine has already embedded the old RLS predicates into the local task execution context during compilation. Recompilation is not triggered during the pipeline connection of batch switching, resulting in the newly entered batch data vectors not being constrained by the new rules. Third, in high-concurrency scenarios, drastic page frame swapping and remapping occur in the off-heap memory buffer. Vectors filtered by old rules and unfiltered new vectors are sent together in contiguous memory for vectorization aggregation and grouping. The entire chain ultimately aggregates the intermediate data processed based on outdated security rules to the coordinating node and generates group reports. This allows highly sensitive business details that should be blocked by lower-level tenants to be secretly included in the aggregated statistics, causing data isolation failure and sensitive data leakage in a multi-tenant environment. The root cause of this problem is that the entire execution path, from optimizer cache reuse and vectorized code generation to off-heap memory swapping, consistently lacks mandatory penetration verification of the runtime execution context by the RLS policy version. Summary of the Invention
[0003] This application discloses a data security isolation method and system based on access permission verification, which aims to solve the problems of data security isolation failure and sensitive data leakage caused by the lack of intermediate result caching and reuse and security rule version verification in multi-tenant distributed computing platforms.
[0004] The technical solution of this application is as follows: This application discloses a data security isolation method based on access permission verification, applied in a multi-tenant distributed computing platform, including: Receive a data access request and parse the requester's tenant identification information and the requested data range from the data access request; Obtain the target data block, which is configured with a row-level access permission bitmap and access permission version information indicating when the row-level access permission bitmap was generated; Get the latest access permission version information; Compare the access permission version information of the target data block with the latest access permission version information to determine whether the access permission version of the target data block is outdated; When the access permission version of the target data block is outdated, identify the difference between the latest access permission and the access permission on which the target data block is based; Based on the difference information, perform batch permission comparison on the data rows in the target data block to generate row-level access permission correction information; The row-level access permission correction information is logically combined with the original row-level access permission bitmap in the target data block, and the row-level access permission bitmap of the target data block is updated in place. Perform row-level access permission verification based on the updated row-level access permission bitmap and the requester tenant identification information; If the validation passes, return the data within the requested range; if the validation fails, return a rejection message.
[0005] This application also discloses a data security isolation system based on access permission verification, applied in a multi-tenant distributed computing platform, including: The data access request receiving module is used to receive a data access request and parse the requesting tenant identification information and the requested data range from the data access request; The data block acquisition module is used to acquire the target data block, which is configured with a row-level access permission bitmap and access permission version information indicating the generation of the row-level access permission bitmap. The version information acquisition module is used to obtain the latest access permission version information; The version comparison module is used to compare the access permission version information of the target data block with the latest access permission version information to determine whether the access permission version of the target data block is outdated. The difference identification module is used to identify the difference between the latest access permission and the access permission on which the target data block is based when the access permission version of the target data block is outdated. The correction information generation module is used to perform batch permission comparison on the data rows in the target data block based on the difference information, and generate row-level access permission correction information; The bitmap update module is used to logically combine the row-level access permission correction information with the original row-level access permission bitmap in the target data block, and update the row-level access permission bitmap of the target data block in place. The permission verification module is used to perform row-level permission verification based on the updated row-level permission bitmap and the requester tenant identification information. The access response module is used to return data within the requested range when the verification passes, and to return a rejection message when the verification fails.
[0006] This application discloses a data security isolation method and system based on access permission verification. It receives data access requests and parses out the requesting tenant identification information and the requested data range to obtain a target data block configured with a row-level access permission bitmap and access permission version information. The method further obtains the latest access permission version information and compares the access permission version of the target data block with the latest version to determine if it is outdated. When the version is outdated, the system identifies the difference between the latest permissions and the permissions on which the target data block is based, and performs batch permission comparison on the data rows based on this difference information to generate row-level access permission correction information. Subsequently, the correction information is logically combined with the original bitmap to update the row-level access permission bitmap of the target data block in-situ. Finally, row-level access permission verification is performed based on the updated bitmap and the requesting tenant identification information, and data or rejection information is returned according to the verification result.
[0007] Instruction manual illustrations Figure 1 This is a flowchart illustrating a data security isolation method based on access permission verification provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of a data security isolation system based on access permission verification provided in an embodiment of the present invention. Detailed Implementation
[0008] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.
[0009] Traditional multi-tenant cloud computing and distributed big data analytics environments typically configure fine-grained access control rules to ensure data security and privacy when processing massive amounts of cross-tenant data. However, during peak business periods, frequent adjustments to compliance policies and changes in business status, as well as the query optimizer's mechanism for reusing intermediate result cache blocks, can lead to the system directly reusing intermediate result caches containing outdated data sets when processing newly entered aggregation subtasks. Furthermore, the underlying vectorized execution engine lacks mandatory penetration verification of security rule versions within the runtime execution context when processing batch data. This allows some data rows not filtered by the latest row-level rules to directly perform vectorized aggregation calculations and grouping concatenation in contiguous off-heap memory, ultimately leading to data security isolation failure and sensitive data leakage.
[0010] In this regard, refer to Figure 1 , Figure 1 This is a flowchart illustrating a data security isolation method based on access permission verification provided in an embodiment of the present invention, applied in a multi-tenant distributed computing platform, including: S11, Receive a data access request, and parse the requesting tenant identification information and the requested data range from the data access request; S12, Obtain the target data block, which is configured with a row-level access permission bitmap and access permission version information indicating the basis for generating the row-level access permission bitmap; S13, obtain the latest access permission version information; S14. Compare the access permission version information of the target data block with the latest access permission version information to determine whether the access permission version of the target data block is outdated. S15, when the access permission version of the target data block is outdated, identify the difference information between the latest access permission and the access permission on which the target data block is based; S16, Based on the difference information, perform batch permission comparison on the data rows in the target data block to generate row-level access permission correction information; S17, logically combine the row-level access permission correction information with the original row-level access permission bitmap in the target data block, and update the row-level access permission bitmap of the target data block in place. S18, perform row-level access permission verification based on the updated row-level access permission bitmap and the requester tenant identification information; S19 returns the data within the requested range if the validation passes; otherwise, it returns a rejection message.
[0011] This application introduces row-level access permission bitmaps and access permission version information, and designs a bitmap update mechanism based on version comparison and difference correction, thereby realizing dynamic and efficient management of data block access permissions and effectively solving the problem of data security isolation failure in multi-tenant environments.
[0012] The method proposed in this application is applied to a multi-tenant distributed computing platform, which typically consists of multiple computing nodes, storage nodes, metadata services, and consistency coordination services. In such an environment, a "tenant" refers to an entity with independent resources and data isolation logic on a shared infrastructure, such as different companies or departments. A "data access request" is an instruction issued by a user or application to the platform to obtain specific data. "Requester tenant identification information" uniquely identifies the tenant initiating the data access request. The "requested data range" clarifies the set of data involved in the request. The "target data block" refers to the smallest unit of data processed or accessed during distributed storage or computing, such as a data partition, a file block, or a data cache block in memory.
[0013] A "row-level access permission bitmap" is a compact data structure where each bit corresponds to a row of data in the target data block, indicating whether that row of data is allowed to be accessed by a specific tenant or under specific conditions. For example, a bit of 1 indicates that access is allowed, while a bit of 0 indicates that access is denied. "Access permission version information" is an identifier used to record the version of the access permission rules on which the row-level access permission bitmap was generated, ensuring the real-time nature and consistency of the access permission rules.
[0014] "Latest Access Permission Version Information" refers to the version identifier of the latest access permission rule currently in effect in the system. "Difference Information" refers to the specific changes between the latest access permission rule and the older version of the access permission rule upon which the target data block is based, such as newly added or removed denial conditions. "Row-Level Access Permission Correction Information" is a bitmap generated through batch permission comparison based on the difference information, used to guide how to correct the original row-level access permission bitmap in the target data block.
[0015] This application aims to address the issue of row-level data security isolation failure caused by dynamic changes in permission rules and the reuse of intermediate data caches in multi-tenant distributed computing platforms. By introducing version control and incremental update mechanisms, it ensures that the access permissions of data blocks are always consistent with the latest rules, thereby preventing the leakage of sensitive data.
[0016] In its implementation, this application first receives a data access request. This request can be initiated by a user through a client or triggered by a computational task within the platform. For example, a data analysis task that needs to access a dataset will generate a data access request. After the request is received, the system parses it to extract the requester's tenant identification information and the requested data range. The requester's tenant identification information can be a tenant ID, a user ID, or any unique identifier associated with the tenant. The requested data range can be a table name, a partition range, a query condition, or a file path.
[0017] The system then retrieves the target data block. This target data block can be a data file stored in a distributed file system or an intermediate result cache block in the computing engine's memory. For example, when a query task needs to process data from a table, it loads the corresponding data block from the storage layer. Each target data block is configured with a row-level access permission bitmap and access permission version information indicating the version on which the bitmap was generated. The row-level access permission bitmap can be a bit array equal to the number of data rows in the data block, with each bit corresponding to one row of data. The access permission version information can be a timestamp, a sequence number, or a hash value, used to identify the version of the permission rules used to generate the bitmap.
[0018] Next, the system will obtain the latest access permission version information. This can be obtained by querying the metadata service or the consistency coordination service. For example, the metadata service maintains the latest access permission rules for all tenants and their corresponding version numbers.
[0019] The system then compares the access permission version information of the target data block with the latest access permission version information to determine whether the access permission version of the target data block is outdated. For example, if the access permission version information of the target data block is a timestamp, and the timestamp of the latest access permission version information is later than the timestamp of the target data block, it indicates that the access permission version of the target data block is outdated.
[0020] When the access permission version of a target data block is outdated, the system identifies the differences between the latest access permission and the access permission on which the target data block is based. For example, by comparing two versions of the permission rule set, newly added and removed denial conditions can be identified.
[0021] Based on this discrepancy information, the system performs batch permission comparisons on the data rows in the target data block, generating row-level access permission correction information. For example, for a newly added denial condition, the system iterates through each row in the target data block, determining whether it meets the condition. If it does, the system marks the row as deprived of access in the correction information. For a removed denial condition, the system determines which rows that were previously denied access can now be accessed.
[0022] Finally, the system logically combines the row-level access permission correction information with the existing row-level access permission bitmap in the target data block, updating the target data block's row-level access permission bitmap in-place. For example, if the correction information indicates that access to a row should be denied, the corresponding bit in the original bitmap is set to 0; if the correction information indicates that access to a row should be allowed, the corresponding bit is set to 1. This in-place update mechanism avoids regenerating the entire data block, improving efficiency.
[0023] Based on the updated row-level access permission bitmap, the system will perform row-level access permission verification by combining the requester's tenant identification information. For example, the system will query the updated bitmap based on the requester's tenant identity to determine whether the requester has the right to access each row of data within the requested data range.
[0024] If the validation passes, the system will return the data within the requested range. If the validation fails, the system will return a rejection message, such as an error code or an access denied message.
[0025] This embodiment operates on a multi-tenant structured analytics cluster. The cluster consists of three roles: Metadata Coordinator Service (MCS), Compute Node Group (CNG), and Object Storage Backend (OSB). Communication links between these roles are encrypted using Transport Layer Security (TLS) 1.3. MCS employs a three-replica deployment based on the Raft protocol (a distributed consensus protocol) and is responsible for maintaining an authoritative view of the access permission version numbers and row-level access permission bitmaps for all target data blocks. CNG comprises several stateless compute nodes and is responsible for executing Structured Query Language (SQL) parsing, bitmap generation, and row-level permission verification. OSB stores the actual data files and associated access permission metadata headers. Tenant authorization data originates from policy sets entered by tenant administrators through the Role-Based Access Control (RBAC) console. After integrity verification, the policy sets are stored in the policy repository, and all operations are logged. Original sensitive fields (ID card number, mobile phone number) undergo one-way hash desensitization processing based on SHA-256 (Secure Hash Algorithm 256 bits) before being stored on disk. The mapping between the hash value and the original value is stored in a trusted vault in encrypted form, and the original value cannot be retrieved from any query path.
[0026] S11, the execution entity is the gateway of the multi-tenant structured analysis cluster. An analyst from tenant A initiates a structured query language query to the gateway via a standard JDBC (Java Database Connectivity) driver. The gateway receives this query as a single data access request at the protocol layer and places it in the gateway's request receiving queue. The request receiving queue adopts a first-in, first-out (FIFO) strategy, with an average latency of approximately 1.2 milliseconds from enqueue to processing start, and approximately 6.5 milliseconds for P99.
[0027] The gateway remains the executing entity. It reads the client identifier (here, the SHA-256 hash of tenant A's client certificate) from the standard JDBC header of the data access request, and then uses a reverse lookup of the local session table to obtain the requesting tenant's identification information, namely tenant A's tenant code (a 32-bit hexadecimal string). Simultaneously, the gateway parses the requested data range from the SQL text of the data access request, specifically the time window (here, the entire year of 2024) and several field conditions (here, a business line field equal to a certain value, and a region identifier field not equal to several blacklist values). After parsing, the requested data range is encapsulated into a logical plan, which is then handled by a compute node in the CNG.
[0028] S12, the execution entity is the compute node that undertakes the logical plan. The compute node locates a target data block D1 based on the requested data range. The storage path of D1 is determined by the table name and partition key corresponding to the data range. The compute node reads the metadata header of D1 from the OSB. The metadata header contains three sets of key fields: first, the Global Unique Identifier (GUID) of D1, which embeds the access permission version number upon which D1 is based; second, the access permission version number V_local upon which D1 is based (represented as a monotonically increasing 64-bit integer); and third, a checksum field (using CRC32C, i.e., Cyclic Redundancy Check 32 bits Castagnoli, 32-bit Castagnoli polynomial cyclic redundancy check, covering the bitmap area and version number of D1). D1 is also configured with a row-level access permission bitmap BITMAP_local. The bitmap corresponds bit to each row in D1; a bit value of 1 indicates that the corresponding row is visible to the current tenant, and a bit value of 0 indicates that the corresponding row is not visible to the current tenant.
[0029] S13, the execution entity is MCS. The compute node submits the globally unique identifier of D1 to MCS. MCS searches in its internal consistency view using the globally unique identifier as the primary key, retrieving the latest published access permission version number V_latest and the corresponding row-level access permission bitmap BITMAP_latest corresponding to that primary key. The MCS search path is as follows: using the high 48 bits of the globally unique identifier as the tenant identifier segment index, the middle 48 bits as the timestamp index, and the low 16 bits as the sequence number range index, a total of 3 random memory accesses, with each access taking approximately 200 nanoseconds, and the overall search taking approximately 600 nanoseconds.
[0030] S14, the execution entity is still MCS. MCS performs a 64-bit unsigned integer comparison: V_local and V_latest. If V_local equals V_latest, the access permission version of D1 is not lagging, and MCS directly returns the current authoritative view (V_latest and BITMAP_latest), and the compute node does not need to enter the differential bitmap correction process; if V_local is less than V_latest, the access permission version of D1 is lagging, and MCS enters the differential bitmap correction process; the case where V_local is strictly greater than V_latest will not occur under the monotonically increasing constraint. If it occurs, MCS will trigger an alarm and discard the metadata header of D1, taking the metadata header of D1 as the standard. This abnormal branch is not involved in this embodiment.
[0031] S15, the execution entity remains MCS. MCS loads the policy set P_latest corresponding to V_latest and the policy set P_local corresponding to V_local from the policy library (the policy library retains historical snapshots by version number; snapshots are stored as key-value pairs, with the key being the version number and the value being the Protobuf serialization result of the policy set; Protobuf is Protocol Buffers, a structured data serialization protocol). MCS performs differential calculations on P_latest and P_local, identifying newly added and removed rejection conditions in P_latest relative to P_local. Newly added rejection conditions form the incremental tightening condition set T_set, and removed rejection conditions form the incremental relaxation condition set R_set. T_set and R_set together constitute the difference information. The differential calculation uses a hash-based (SHA-256) policy identifier comparison. The time consumed in determining additions and removals increases linearly with the number of policies. With a policy count of 1×10... 4 The differential calculation for the example bar takes approximately 8 milliseconds.
[0032] S16, the execution entity remains MCS. For each incremental tightening condition in T_set, MCS calculates the set of data rows in D1 that the condition hits, based on the field metadata of D1, and sets the hit position to 1 and the miss position to 0, forming a tightening bitmap BITMAP_tight. For each incremental relaxation condition in R_set, MCS calculates the set of data rows that the condition was originally rejected in D1, based on the field metadata of D1, and sets the originally rejected position to 1 and the remaining positions to 0, forming a relaxation bitmap BITMAP_relax. MCS then encapsulates BITMAP_tight and BITMAP_relax into row-level access permission correction information. Batch permission comparison uses vectorized execution, with the dictionary distribution in the field metadata serving as input for condition evaluation, 1×102 6 The overall time to generate BITMAP_tight and BITMAP_relax at row size is approximately 18 milliseconds.
[0033] S17, the execution entity is still MCS. MCS executes the logic combination in two steps, applying a mutex lock and a bitmap operation lock for two layers of serialization. Step 1: Acquire the mutex lock corresponding to D1, read the row-level access permission bitmap BITMAP_local stored in D1, perform a bitwise AND operation between BITMAP_local and BITMAP_tight to obtain the intermediate access permission bitmap BITMAP_mid, write BITMAP_mid to the bitmap area of D1, and release the mutex lock. Step 2: Acquire the bitmap operation lock corresponding to D1, read the bitmap area of D1 (at this time the content is BITMAP_mid), perform a bitwise OR operation between BITMAP_mid and BITMAP_relax to obtain the final row-level access permission bitmap BITMAP_new, update D1 with BITMAP_new in the bitmap area of D1, and release the bitmap operation lock. Bitmap update requests from different compute nodes on the same D1 node enter the bitmap operation queue. The queue adopts a first-in, first-out (FIFO) strategy. The request at the head of the queue acquires the lock and begins execution. After execution, it releases the lock and notifies the initiator of the request at the head of the queue. Upon receiving the notification, the initiator removes the request from the queue. 1×10 6 The two-step process of bitwise AND and bitwise OR at row size takes approximately 6 milliseconds.
[0034] S18, the execution entity is the compute node that undertakes the logical plan. MCS encapsulates BITMAP_new and V_latest into a modified access permission metadata header and returns it to the compute node. The compute node verifies the integrity of the bitmap using a CRC32C checksum. After successful verification, it writes the modified metadata header in-place to the metadata header position of D1 and retains a copy of BITMAP_new in memory. The compute node holds a copy of BITMAP_new and the requesting tenant identification information (here, the tenant code of tenant A) in memory. For each row required to be scanned in the logical plan, it reads the corresponding bit from BITMAP_new using the row offset as an index. A bit value of 1 indicates that the row is visible to the requesting tenant, and a bit value of 0 indicates that the row is not visible to the requesting tenant. The single-bit lookup time is approximately 5 nanoseconds, 1×10 6 The overall time for row-level permission verification during a full table scan with row size is approximately 5 milliseconds.
[0035] In scenario S19, if the verification passes (at least one visible row exists within the selected data range), the compute node projects the selected visible data rows into a result set according to the requested fields. The result set is then returned to the gateway via a standard JDBC driver, and the gateway then sends it back to tenant A's analyst client. The result set undergoes field-level anonymization verification on the gateway side (e.g., performing SHA-256 hash value verification on the ID number field to ensure consistency between the hash values and the data stored on disk). After successful verification, it is sent to the client. The P99 transmission time for the result set under 200 concurrent requests is approximately 23 milliseconds.
[0036] In scenarios where validation fails (no visible rows exist within the selected data range), the compute node encapsulates a rejection message into a standard response (status code 403, i.e., Forbidden) and returns it to the gateway. The gateway then forwards the rejection message to the client. The rejection message is a machine-readable JSON object containing three fields: error code (ROW_LEVEL_ACCESS_DENIED in this case), error description ("No data rows visible to the current tenant within the selected data range"), and tracking identifier (the request tracking number returned by MCS, a 32-bit hexadecimal string). The client visualization layer presents the rejection message as a readable prompt. The prompt content is localized by the client; this embodiment does not involve the client's localization logic.
[0037] Quantifiable technical effects: The deployment scale is 1000 tenants, with an average of 1×10 per tenant. 4 Tables, each table averaging 1×10 6At the row level, the P99 latency (99th percentile response time, i.e., the upper limit of request time at the 99th percentile) for row-level permission verification under 200 concurrent requests (the concurrency number is set by the load testing tool wrk) decreased from 480 milliseconds based on real-time calculation by the rule engine to 47 milliseconds, a relative reduction of approximately 90.2%; under 500 concurrent requests, QPS (Queries Per Second) increased from 3200 to 18200, a relative improvement of approximately 468%; 1×10 6 The single-bit query time on the target data block is approximately 5 nanoseconds, and the overall time for row-level permission verification during a full table scan is approximately 5 milliseconds. The end-to-end latency for broadcasting access permission version updates to all target data blocks to complete bitmap correction is less than 800 milliseconds in 99% of scenarios, which is approximately 86.7% lower than the maximum 10-minute inconsistency window of the periodic full table recalculation scheme. The average bitmap storage overhead is approximately 122KB per target data block, and approximately 2×10^6 bits are stored per node in the object storage backend. 4 With one target data block, the bitmap occupies approximately 2.44GB in total, accounting for about 1.91% of the node's memory (taking 128GB as an example), which is within an acceptable range.
[0038] In response, this application further proposes a more efficient and refined bitmap update strategy, which achieves fast in-situ update of row-level access permission bitmaps by identifying permission differences and generating incremental correction bitmaps.
[0039] The steps described above, which logically combine the row-level access permission correction information with the original row-level access permission bitmap in the target data block to update the row-level access permission bitmap in-place, include: Obtain the latest access permissions and the access permissions on which the target data block is based; Identify the newly added denial conditions in the latest access permissions relative to the access permissions on which the target data block is based, and form incremental tightening conditions; Identify the rejection conditions in the latest access permissions that are related to the removal of the access permissions on which the target data block is based, and form incremental relaxation conditions; Based on the incremental tightening condition, perform batch permission comparison on the data rows in the target data block to generate a tightening bitmap; Based on incremental relaxation conditions, batch permission comparisons are performed on data rows in the target data block to generate a relaxation bitmap. Perform a bitwise AND operation between the original row-level access permission bitmap and the tightened bitmap in the target data block to obtain an intermediate access permission bitmap; Perform a bitwise OR operation between the intermediate permission bitmap and the relaxed bitmap to obtain the final row-level access permission bitmap; Update the final row-level access permission bitmap in place to the target data block.
[0040] Specifically, when performing a bitmap update operation, it is first necessary to obtain the latest access permission rule set and the access permission rule set on which the target data block was generated. These rule sets can contain a series of conditions used to define which tenants or users can access which data rows.
[0041] The process of identifying new denial conditions in the latest access permissions relative to the access permissions upon which the target data block is based, forming incremental tightening conditions, involves comparing the old and new permission rules to find conditions that are explicitly prohibited under the latest permissions but might have been allowed under the old permissions. These conditions will cause the access permissions for certain data rows to change from allowed to denied, i.e., permission tightening. For example, if the old permissions allowed all tenants to access a certain type of data, while the new permissions prohibit access for a specific tenant, then the prohibition condition for that specific tenant is the incremental tightening condition.
[0042] Simultaneously, identifying the denial conditions in the latest access permissions relative to the access permissions upon which the target data block is based, forming incremental relaxation conditions, refers to finding those conditions that are no longer prohibited in the latest permissions but were prohibited in the old permissions. These conditions will cause the access permissions of certain data rows to change from denied to permitted, i.e., permission relaxation. For example, if the old permissions prohibited a specific tenant from accessing a certain type of data, and the new permissions remove this restriction, then the removal of this restriction is an incremental relaxation condition.
[0043] Based on the established incremental tightening conditions, batch permission comparisons are performed on the data rows in the target data block to generate a tightening bitmap. The tightening bitmap is a binary bitmap corresponding to the number of data rows in the target data block. If a data row should be denied access due to the incremental tightening conditions, its corresponding bit is set to 0 (indicating denial); otherwise, it is set to 1 (indicating permission). This process can be efficiently completed in batches, avoiding row-by-row rule checks.
[0044] Similarly, based on the resulting incremental relaxation conditions, batch permission comparisons are performed on the data rows in the target data block to generate a relaxation bitmap. The relaxation bitmap is also a binary bitmap, in which the corresponding bit is set to 1 (indicating permission) if a data row should be allowed access due to the incremental relaxation conditions; otherwise, it is set to 0 (indicating denial).
[0045] Subsequently, a bitwise AND operation is performed between the original row-level access permission bitmap and the tightened bitmap in the target data block to obtain an intermediate permission bitmap. The logic of the bitwise AND operation is that the result bit is 1 only if the corresponding bits in both the original bitmap and the tightened bitmap are 1. This means that any data row whose access is denied by the incremental tightening condition (the corresponding bit in the tightened bitmap is 0) will have its permissions in the original bitmap tightened to denied.
[0046] Finally, a bitwise OR operation is performed between the intermediate permission bitmap and the relaxation bitmap to obtain the final row-level access permission bitmap. The logic of the bitwise OR operation is that if any corresponding bit in the intermediate permission bitmap or the relaxation bitmap is 1, the result bit will be 1. This means that any data row that is allowed access by the incremental relaxation condition (with a corresponding bit of 1 in the relaxation bitmap) will have its permission relaxed to allowed in the intermediate permission bitmap.
[0047] Finally, the obtained row-level access permission bitmap is updated in-place to the target data block, thus completing the correction of the access permission bitmap.
[0048] The solution proposed in this application decomposes complex permission rule changes into two independent parts: incremental tightening and incremental relaxation, and generates corresponding correction bitmaps for each part, thereby achieving efficient updating of the row-level access permission bitmap.
[0049] In some preferred embodiments, a specific example is given below. Suppose a target data block contains 10 rows of data, and its original row-level access permission bitmap is [1, 1, 1, 1, 0, 1, 1, 1, 1, 1], indicating that all rows except the 5th row are accessible. Now, the access permission rules change: 1. Add a new rejection condition: Tenant A cannot access lines 3 and 7.
[0050] 2. Remove a denial condition: Tenant A can now access line 5 (assuming there was a previous rule that prohibited access to line 5).
[0051] Based on the above changes, the system first obtains the latest access permissions and the access permissions on which the target data block is based.
[0052] Then, the incremental tightening condition is identified as "tenant A cannot access rows 3 and 7", and a tightening bitmap is generated based on this. After performing batch permission comparisons on the data rows in the target data block, the tightening bitmap is generated as [1, 1, 0, 1,1, 1, 0, 1, 1, 1] (the 3rd and 7th bits are 0, indicating rejection).
[0053] Next, the incremental relaxation condition is identified as "tenant A can access row 5", and a relaxation bitmap is generated based on this. After performing batch permission comparisons on the data rows in the target data block, the relaxation bitmap is generated as [0, 0, 0, 0, 1, 0,0, 0, 0, 0] (only the 5th bit is 1, indicating that it is allowed).
[0054] Subsequently, a bitwise AND operation is performed on the original row-level access permission bitmap [1, 1, 1, 1, 0, 1, 1, 1, 1, 1] in the target data block and the tightened bitmap [1, 1, 0, 1, 1, 1, 0, 1, 1, 1] to obtain the intermediate access permission bitmap [1, 1, 0, 1, 0, 1, 0, 1, 1, 1].
[0055] Finally, perform a bitwise OR operation on the intermediate permission bitmap [1, 1, 0, 1, 0, 1, 0, 1, 1, 1] and the relaxed bitmap [0, 0, 0, 0, 1, 0, 0, 0, 0, 0] to obtain the final row-level access permission bitmap [1, 1, 0, 1, 1, 1, 0, 1, 1, 1].
[0056] Ultimately, the bitmap is updated in place to the target data block, indicating that rows 3 and 7 are denied access, while row 5 is allowed access, and the remaining rows remain allowed access, thus achieving efficient and accurate permission updates.
[0057] This application further proposes a step of performing a bitwise AND operation between the original row-level access permission bitmap in the target data block and the tightened bitmap to obtain an intermediate access permission bitmap, including: Add the bitmap operation requests to be executed to the bitmap operation queue; Acquire the mutex lock corresponding to the target data block; Read the row-level access permission bitmap of the target data block; Perform a bitwise AND operation between the row-level access permission bitmap and the tightened bitmap to obtain an intermediate access permission bitmap; Write the intermediate permission bitmap into the bitmap area of the target data block; Release the mutex.
[0058] Specifically, a bitmap operation request refers to any operation request that needs to modify or read the row-level access permission bitmap of the target data block, such as permission tightening, permission loosening, or query requests. These requests are uniformly added to the bitmap operation queue before being processed to achieve orderly management and scheduling of concurrent operations. A mutex lock is a synchronization mechanism used to ensure that only one thread or process can access the row-level access permission bitmap of the target data block at any given time. By acquiring the mutex lock corresponding to the target data block, exclusive access rights can be provided for subsequent bitmap read, modification, and write operations, thereby avoiding concurrent conflicts. After acquiring the mutex lock, reading the row-level access permission bitmap of the target data block will obtain the latest bitmap state, ensuring that subsequent bitwise AND operations are based on an accurate benchmark. Subsequently, performing a bitwise AND operation between the row-level access permission bitmap and the tightening bitmap to obtain an intermediate permission bitmap is the core permission tightening logic. It performs a logical AND operation between existing permissions and tightening conditions to remove permissions that are no longer allowed. After calculating the intermediate permission bitmap, the operation of writing the intermediate permission bitmap to the bitmap region of the target data block persists the updated bitmap to the target data block. Finally, the mutex lock release operation releases the exclusive access to the bitmap, allowing other waiting requests to acquire the lock and continue execution, thereby ensuring efficient resource utilization and system responsiveness.
[0059] This application's solution effectively addresses the data consistency problem of bitmap update operations in concurrent environments by introducing a bitmap operation queue and a mutex lock mechanism. Through the above technical solution, this application can effectively resolve the issues of row-level access permission bitmap data inconsistency and data corruption that may occur during concurrent update operations in multi-tenant distributed computing platforms.
[0060] In some preferred embodiments, a specific example is given below. Suppose that in a multi-tenant distributed computing platform, there exists a target data block whose row-level access permission bitmap needs to be tightened. Simultaneously, two independent permission management services (or threads) initiate permission tightening requests for this data block almost at the same time.
[0061] Without the solution described in this application, the two requests might simultaneously read the initial state of the bitmap, then each calculate the tightened bitmap and attempt to write to it. This could result in one request's update being overwritten by the other, leading to an incorrect final bitmap state and failing to fully achieve all the expected permission tightening.
[0062] However, when using the scheme of this application, when the first access control service initiates a tightening request, the request is added to the bitmap operation queue for the target data block. Subsequently, the service attempts to acquire the mutex lock associated with the target data block. Once the lock is successfully acquired, it exclusively reads the current row-level access permission bitmap, performs a bitwise AND operation, and writes the updated intermediate access permission bitmap to the target data block. After completing the operation, the mutex lock is released.
[0063] Meanwhile, the tightening request initiated by the second permission management service will also be added to the same bitmap operation queue and wait for the mutex lock to be released. Once the first service releases the lock, the second service can acquire the lock, then read the latest bitmap state that has been updated by the first service, perform its own bitwise AND operation, and write the final intermediate permission bitmap.
[0064] This mechanism ensures that even when multiple concurrent requests arrive simultaneously, they can update the row-level access permission bitmap sequentially and atomically, protected by a mutex lock, in a queue-like order. This guarantees that each update is based on the latest bitmap state, avoiding data races and inconsistencies, thus ensuring the accuracy of permission tightening operations and the integrity of data security isolation.
[0065] This application further proposes the following steps for performing a bitwise OR operation between the intermediate permission bitmap and the relaxed bitmap to obtain the final row-level access permission bitmap: Acquire the bitmap operation lock corresponding to the target data block; Read the row-level access permission bitmap of the target data block; Perform a bitwise OR operation between the row-level access permission bitmap and the relaxation bitmap to obtain the final row-level access permission bitmap; Update the final row-level access permission bitmap in-place to the target data block; Release the bitmap operation lock.
[0066] Specifically, a bitmap operation lock can be understood as a synchronization mechanism, such as a mutex or read-write lock. Its purpose is to ensure that at any given time, only one thread or process can modify the row-level access permission bitmap of a specific target data block. By acquiring this lock, exclusive access to and modification of the bitmap can be achieved, thus avoiding data inconsistency issues caused by concurrent modifications. Before performing a bitmap update operation, the bitmap operation lock associated with the target data block to be updated must first be acquired. Once the lock is successfully acquired, the currently stored row-level access permission bitmap in the target data block can be safely read. Subsequently, a bitwise OR operation is performed between the read row-level access permission bitmap and a pre-generated relaxation bitmap. The purpose of the bitwise OR operation is to merge the permission relaxation information indicated in the relaxation bitmap into the current permission bitmap, thereby obtaining a final row-level access permission bitmap reflecting the latest permission rules. After completing the bitwise OR operation and generating the final row-level access permission bitmap, this final bitmap is updated in-situ to the bitmap storage area of the target data block. In-situ updates mean directly modifying the data in the existing bitmap of the target data block without creating a new data block or performing large-scale data migration. Finally, after the bitmap update operation is complete, the previously acquired bitmap operation lock must be released so that other waiting threads or processes can acquire the lock and perform their respective bitmap operations.
[0067] The solution proposed in this application effectively solves the race conditions and data inconsistency problems that may occur when updating row-level access permission bitmaps in a concurrent environment by introducing bitmap operation locks.
[0068] In some preferred embodiments, suppose a target data block exists in a multi-tenant distributed computing platform, and its row-level access permission bitmap needs to be updated according to the latest permission rules. Simultaneously, two independent permission management services (e.g., one service handles permission changes for user A, and the other handles permission changes for user B) initiate update requests for the target data block's bitmap almost simultaneously. Both requests involve performing a bitwise OR operation between their respective generated relaxed bitmaps and the target data block's existing bitmap.
[0069] Without bitmap operation locks, the following may occur: 1. Service A reads the initial state V0 of the bitmap.
[0070] 2. Service B also reads the initial state V0 of the bitmap.
[0071] 3. Service A calculates V_A_new based on V0 and its relaxed bitmap L_A, and attempts to write to it.
[0072] 4. Service B calculates V_B_new based on V0 and its relaxed bitmap L_B, and attempts to write to it.
[0073] If service A completes its write before service B does, service B's write will overwrite service A's update, causing the loss of service A's permission changes. The reverse is also true. This results in an incorrect bitmap state that fails to accurately reflect all effective permission changes.
[0074] After adopting the solution of this application, the situation will become: 1. Service A initiates an update request, attempting to acquire a bitmap operation lock corresponding to the target data block. Assume service A successfully acquires the lock.
[0075] 2. Under the protection of the lock, service A reads the initial state V0 of the bitmap and performs a bitwise OR operation with its relaxed bitmap L_A to obtain V_A_new.
[0076] 3. Service A updates V_A_new in place to the target data block.
[0077] 4. Service A releases the bitmap operation lock.
[0078] 5. While service A holds the lock, service B initiates an update request, attempting to acquire the bitmap operation lock. However, since the lock is already held by service A, service B will be blocked until service A releases the lock.
[0079] 6. After service A releases the lock, service B successfully acquires the lock.
[0080] 7. Under the protection of the lock, service B reads the current state of the bitmap (which is now V_A_new) and performs a bitwise OR operation with its relaxed bitmap L_B to obtain V_B_new.
[0081] 8. Service B updates V_B_new in place to the target data block.
[0082] 9. Service B releases the bitmap operation lock.
[0083] In this way, even if multiple services concurrently request updates, the bitmap modification operations will be executed serially, ensuring that each update is based on the latest bitmap state, thereby guaranteeing the accuracy and consistency of the final bitmap and effectively avoiding data conflicts and permission errors.
[0084] The steps described above for obtaining a target data block, wherein the target data block is configured with a row-level access permission bitmap and access permission version information indicating the basis for generating the row-level access permission bitmap, include: When a computing node generates or receives a target data block, it obtains the globally unique identifier and access permission metadata header of the target data block. Submit the globally unique identifier and the access permission metadata header to the consistency coordination service; The consistency coordination service maintains a consistent view of data block metadata, receives access permission update broadcasts, and updates the latest access permission version number maintained internally. When the target data block is replicated or cached between nodes, the receiving node initiates a query request to the consistency coordination service to obtain the latest access permission version number of the target data block and the corresponding row-level access permission bitmap. The consistency coordination service compares the version number currently carried by the target data block with the latest access permission version number maintained internally. If the version number currently carried by the target data block is outdated, the consistency coordination service performs differential bitmap correction logic on the metadata of the target data block to generate a corrected row-level access permission bitmap and the latest version number; The consistency coordination service returns the corrected row-level access permission bitmap and the latest version number to the receiving node; The receiving node writes the corrected metadata in-place into the access permission metadata header of the target data block.
[0085] Specifically, when a compute node generates a new target data block, it needs to assign it a globally unique identifier and initialize its access permission metadata header. This metadata header typically contains a row-level access permission bitmap and information indicating the access permission version on which the bitmap was generated. When a compute node receives a data block from another node, it also needs to obtain its globally unique identifier and access permission metadata header. The globally unique identifier is used to uniquely identify a data block throughout the distributed system, ensuring its uniqueness. The access permission metadata header carries the data block's permission status and is the basis for permission verification.
[0086] Furthermore, the obtained globally unique identifier and access permission metadata header are submitted to the consistency coordination service. The consistency coordination service can be understood as a centralized or distributed consistency component, such as a coordination service implemented based on ZooKeeper, Etcd, or Raft protocols. Its core function is to maintain a consistent view of all data block metadata, ensuring that in a distributed environment, all nodes have a unified and up-to-date understanding of the data block's permission metadata. This service receives access permission update broadcasts from the permission management system and updates its internally maintained latest access permission version number accordingly.
[0087] When a target data block is replicated or cached between different nodes, the receiving node, after acquiring the data block, initiates a query request to the consistency coordination service. The purpose of this request is to obtain the latest access permission version number and the corresponding row-level access permission bitmap for the data block. Upon receiving the query request, the consistency coordination service compares the version number currently carried by the target data block with the latest access permission version number maintained internally. If it finds that the version number currently carried by the target data block is outdated, meaning the data block's permission metadata is not up-to-date, the consistency coordination service will perform differential bitmap correction logic on the target data block's metadata. This logic, based on version differences, calculates the necessary corrections to the original row-level access permission bitmap and generates a corrected row-level access permission bitmap and the latest version number. Finally, the consistency coordination service returns the corrected row-level access permission bitmap and the latest version number to the receiving node that initiated the query. After receiving this corrected metadata, the receiving node writes it in-place into the access permission metadata header of the target data block, thereby ensuring that the data block has the latest and consistent permission metadata locally.
[0088] The solution proposed in this application effectively solves the problem of difficulty in guaranteeing the consistency of data block permission metadata in a distributed environment by introducing a consistency coordination service.
[0089] In some preferred embodiments, a specific example is given below. Assume a multi-tenant distributed computing platform includes multiple computing nodes A, B, and C, and a consistency coordination service.
[0090] 1. Data Block Generation and Registration: Compute node A generates a target data block named "Order Data Block" and assigns it a globally unique identifier "OrderID_001". Simultaneously, it initializes the access permission metadata header for this data block, which contains a row-level access permission bitmap generated based on the current permission version V1. Compute node A submits "OrderID_001" and the access permission metadata header to the consistency coordination service. The consistency coordination service records the metadata of "OrderID_001" and marks its current version as V1.
[0091] 2. Permission Update Broadcast: At this point, the platform administrator has modified the access permissions for order data, for example, by adding a rejection condition. The permission management system broadcasts this update to the consistency coordination service. Upon receiving the update, the consistency coordination service updates its internally maintained latest access permission version number to V2.
[0092] 3. Data Block Replication and Query: Compute node B needs to process the "Order Data Block," so it replicates this data block from compute node A. The replicated data block still carries the metadata of version V1. To ensure the up-to-dateness of permissions, compute node B initiates a query request to the consistency coordination service, providing "OrderID_001" and its current version V1.
[0093] 4. Version Comparison and Correction: Upon receiving a request, the consistency coordination service compares the current version V1 carried by "OrderID_001" with the latest internally maintained version V2. It finds that V1 lags behind V2. The consistency coordination service then identifies the permission differences between V1 and V2 and performs differential bitmap correction logic on the metadata of the "order data block" based on these differences. For example, if V2 adds a rejection condition compared to V1, the consistency coordination service generates a tightened bitmap, performs a logical AND operation with the original row-level access permission bitmap to obtain the corrected row-level access permission bitmap, and updates its version number to V2.
[0094] 5. Metadata Update: The consistency coordination service returns the revised row-level access permission bitmap and the latest version V2 to compute node B. Upon receiving this information, compute node B writes it in-place into the access permission metadata header of the "Order Data Block". At this point, the "Order Data Block" on compute node B has the latest and consistent permission metadata, which can be used for subsequent row-level access permission verification.
[0095] This application further proposes the following steps for obtaining the globally unique identifier and access permission metadata header of the target data block: When a computing node generates a target data block, it obtains the globally unique identifier of the target data block from the globally unique identifier generator; Bind the globally unique identifier to the access permission metadata header of the target data block; When a receiving node receives a target data block, it verifies the validity of the globally unique identifier carried by the target data block. If the verification passes, the globally unique identifier and the access permission metadata header are submitted to the consistency coordination service; The globally unique identifier is encoded with the access permission version information on which the target data block is based; the consistency coordination service directly indexes the latest access permission version and differential bitmap corresponding to the globally unique identifier.
[0096] Specifically, when a computing node needs to generate a new target data block, it requests a globally unique identifier generator to obtain a unique identifier. This globally unique identifier generator can be a centralized service responsible for allocating and managing identifiers throughout the distributed computing platform, ensuring their global uniqueness. After obtaining the globally unique identifier, it is bound to the access permission metadata header of the target data block, forming a complete metadata structure. This binding operation ensures a strong association between the data block's identity and its permission metadata.
[0097] When a compute node receives a target data block replicated or cached from another node, it first needs to verify the validity of the globally unique identifier carried by the target data block. This verification process may include checking whether the identifier's format conforms to the specification, whether it exists in a known identifier registry, or confirming its integrity through methods such as checksums. If the verification passes, the globally unique identifier and the corresponding access permission metadata header are submitted to the consistency coordination service.
[0098] In a preferred implementation, the globally unique identifier is designed to embed the access permission version information upon which the target data block is based. This means that when generating the globally unique identifier, the version number or other version indication information of the access permission associated with the current data block is directly encoded into a specific field or bit segment of the identifier. For example, a portion of the identifier can be specifically used to store the access permission version number. Therefore, when the consistency coordination service receives the globally unique identifier, it can directly extract the access permission version information it is based on from the identifier itself without performing additional query or parsing operations. Based on this embedded version information, the consistency coordination service can efficiently and directly index the corresponding latest access permission version and the related differential bitmap, thereby quickly performing permission comparison and correction.
[0099] The solution in this application achieves tight coupling between data block identity and permission version information by directly embedding the access permission version information on which the target data block is based into the encoding of its globally unique identifier.
[0100] In some preferred embodiments, a specific example is given below. Assume the globally unique identifier uses a 128-bit structure, where specific bits are reserved for encoding access permission version information. For example, bits 64 to 79 (a total of 16 bits) of the identifier can be designated as the access permission version field. When a compute node generates a data block, it obtains a base identifier from the globally unique identifier generator and writes the access permission version number (e.g., an incrementing integer) of the current data block into the access permission version field of that identifier. When the metadata of this data block is submitted to the consistency coordination service, the service can directly parse the 128-bit identifier and extract the access permission version information from bits 64 to 79. Based on this version information, the consistency coordination service can directly look up the corresponding latest permission rules and differential bitmap in the internally maintained access permission version index, thereby quickly determining whether the data block's permissions are outdated and executing the corresponding correction logic. This approach makes the acquisition and indexing process of access permission version information extremely efficient and direct.
[0101] This application further proposes that when a computing node generates a target data block, the steps for obtaining the globally unique identifier of the target data block from the globally unique identifier generator include: When a compute node needs to generate a globally unique identifier for a target data block, it sends a request to the master identifier generation service. If the primary identifier generation service does not return an identifier within a preset response time, the computing node switches to the local backup identifier generation module. The backup identifier generation module generates a temporary identifier based on the machine identification information of the computing node, the current timestamp, and the incrementing local sequence number; The temporary identifier carries a tag indicating that it was generated for backup; When the primary identifier generation service is restored to availability, the compute node submits all temporary identifiers and corresponding data block information generated by the backup identifier generation module to the primary identifier generation service to perform registration and global uniqueness verification. If the primary identifier generation service detects a conflict in the temporary identifier, it reassigns a globally unique identifier to the conflicting data block and notifies the computing node of the updated identifier.
[0102] Specifically, when a compute node needs to assign a globally unique identifier to a newly generated target data block, it first attempts to send a request to the primary identifier generation service. This primary identifier generation service is typically a centralized, highly available service responsible for assigning unique identifiers throughout the distributed platform. To handle potential transient failures or network latency in the primary service, compute nodes set a preset response time. If the primary identifier generation service fails to return the required identifier within this preset response time, the compute node is triggered to switch to a local backup identifier generation module.
[0103] The local standby identifier generation module is a lightweight component deployed on each compute node, designed to provide temporary identifier generation capabilities when the primary service is unavailable. This module generates temporary identifiers based on the compute node's machine identification information, the current timestamp, and an incrementing local sequence number. The machine identification information can be the compute node's IP address, hostname, or unique ID, ensuring local uniqueness of identifiers generated on the same machine. The current timestamp provides temporal differentiation, while the incrementing local sequence number further guarantees uniqueness when multiple identifiers are generated within a very short timeframe. The resulting identifier is called a temporary identifier, characterized by carrying a flag indicating that it was generated as a standby identifier. This flag can be a specific bit, prefix, or field used to distinguish the temporary identifier from the globally unique identifier generated by the primary service and to indicate the need for subsequent registration and verification.
[0104] In practical applications, when the master identifier generation service becomes available again, compute nodes submit all temporary identifiers generated by the backup module and their corresponding data block information to the master identifier generation service. The master identifier generation service performs registration and global uniqueness verification. Registration means incorporating these temporary identifiers into the management scope of the master service, ensuring their traceability globally. Global uniqueness verification means the master service checks whether these temporary identifiers conflict with already assigned global identifiers. If the master identifier generation service detects any conflicting temporary identifiers—for example, two different compute nodes generating the same temporary identifier during the master service's unavailability, or a temporary identifier duplicating an identifier already assigned by the master service—the master service will reassign new globally unique identifiers for these conflicting data blocks and notify the corresponding compute nodes of the updated identifiers. Upon receiving the updated identifiers, the compute nodes apply them to the corresponding data blocks, thus ensuring that all data blocks ultimately have globally unique identifiers.
[0105] The solution proposed in this application effectively solves the problem of single point of failure or momentary unavailability that may occur in the primary identifier generation service by introducing a local backup identifier generation module and a subsequent registration verification mechanism.
[0106] In some preferred embodiments, a specific example is given below. Assume that in a multi-tenant distributed computing platform, compute node A needs to generate a new data block. It first requests a globally unique identifier from the primary identifier generation service. If the primary identifier generation service does not return an identifier within a preset response time of 500 milliseconds, compute node A immediately switches to its local backup identifier generation module. This module combines compute node A's machine identification information (e.g., 'node-A-001'), the current timestamp (e.g., '20231027103000'), and a local incrementing sequence number (e.g., '001') to generate a temporary identifier, such as 'TEMP-node-A-001-20231027103000-001'. The 'TEMP' prefix in this temporary identifier is the backup generation marker. Compute node A uses this temporary identifier to continue processing the generation of the data block.
[0107] Subsequently, the primary identifier generation service resumed normal operation. Once compute node A detects that the primary service is available, it submits all temporary identifiers generated during the primary service's unavailability (e.g., 'TEMP-node-A-001-20231027103000-001', 'TEMP-node-A-001-20231027103000-002', etc.) along with their corresponding data block information to the primary identifier generation service for registration and verification. Upon receiving these temporary identifiers, the primary service checks for conflicts with already assigned global identifiers. For example, if 'TEMP-node-A-001-20231027103000-001' conflicts with a temporary identifier generated by another compute node B within the same time period or with an identifier already assigned by the primary service, the primary service will reassign a new globally unique identifier for this conflicting data block, such as 'GLOBAL-XYZ-789', and notify compute node A of this update. After receiving the notification, compute node A will update the identifier of the data block from a temporary identifier to a new globally unique identifier, thereby ensuring that all data blocks eventually have a globally unique and conflict-free identifier.
[0108] This application further proposes that when a computing node generates a target data block, the steps for obtaining the globally unique identifier of the target data block from the globally unique identifier generator include: When a compute node needs to generate a globally unique identifier for a target data block, it obtains the tenant identification information of the current operation. Submit the tenant identification information to the globally unique identifier generator; The globally unique identifier generator receives the tenant identification information and obtains the corresponding tenant-specific identifier segment from the preset tenant identifier allocation rules according to the tenant identification information; The globally unique identifier generator combines the current time information and the incrementing sequence number within the tenant-specific identifier segment to generate a globally unique identifier for the target data block. The globally unique identifier generator returns the globally unique identifier of the generated target data block to the computing node.
[0109] Specifically, when a compute node needs to assign a globally unique identifier to a newly generated target data block, it first obtains the tenant identification information of the currently executing operation. This tenant identification information can be the tenant's unique ID, name, or other credentials that uniquely identify the tenant. This information is then submitted to the globally unique identifier generator. Upon receiving the tenant identification information, the globally unique identifier generator allocates one or more tenant-specific identifier segments to the tenant according to preset tenant identifier allocation rules. These identifier segments are pre-planned to ensure that identifiers from different tenants are logically isolated from each other, avoiding conflicts and facilitating management. For example, the allocation rule can be defined as each tenant having a specific prefix or range. After obtaining the tenant-specific identifier segment, the globally unique identifier generator combines the current time information (e.g., a timestamp accurate to milliseconds or microseconds) and an incrementing local sequence number within that segment to generate the final globally unique identifier. The time information ensures the identifier's uniqueness in time, while the incrementing sequence number resolves the conflict problem of multiple identifiers that may be generated at the same point in time. Finally, the generated globally unique identifier is returned to the requesting compute node to bind the access permission metadata header of the target data block.
[0110] The solution proposed in this application incorporates tenant identification information into the generation process of globally unique identifiers, thus enabling the generated identifiers to inherently carry tenant affiliation attributes. Through this technical solution, this application effectively addresses the management complexity and performance overhead issues that may arise from traditional generic identifier generation methods in multi-tenant environments.
[0111] As a specific implementation, suppose there are tenant A and tenant B in a multi-tenant distributed computing platform. When a computing node of tenant A needs to generate a globally unique identifier for a target data block, the computing node first obtains the identification information of tenant A (e.g., "TenantA_ID"). This identification information is submitted to the globally unique identifier generator. The globally unique identifier generator, according to a preset tenant identifier allocation rule, for example, specifies that the identifier range for tenant A is "0x1000-0x1FFF" and the identifier range for tenant B is "0x2000-0x2FFF". At this time, the generator will combine the current timestamp (e.g., a number represented as YYYYMMDDHHMMSSmmm) and an incrementing sequence number (e.g., 001) from the tenant A's exclusive identifier range "0x1000-0x1FFF" to generate a globally unique identifier, the structure of which may be similar to "0x1000_YYYYMMDDHHMMSSmmm_001". When tenant B's compute node also needs to generate an identifier, its identifier will be generated from the "0x2000-0x2FFF" segment, for example, "0x2000_YYYYMMDDHHMMSSmmm_001". In this way, even at the same point in time, data block identifiers generated by different tenants can be clearly distinguished by their prefixes or specific segments, maintaining uniqueness globally. When the consistency coordination service receives these identifiers, it can quickly identify the tenant to which the data block belongs and perform corresponding metadata management and permission processing based on the tenant information, greatly improving processing efficiency and system scalability.
[0112] This application further proposes that, in the above method, the row-level access permission verification is performed based on the pre-calculated row-level access permission bitmap, without the need to calculate the access permission rules in real time at the time of request; the size of the bitmap is linearly related to the number of data rows in the target data block, and does not increase with the content size of the target data block.
[0113] Specifically, a pre-computed row-level access permission bitmap refers to a bitmap where complex access permission rules are converted into a binary bitmap form during data block generation or access permission updates, where each bit represents the access permission status of a data row. This pre-computed mechanism avoids the need to evaluate and match all relevant access permission rules in real time every time a data access request arrives.
[0114] The fact that access permission rules don't need to be calculated in real-time upon request means that when a data access request is received, the system can directly determine the requesting tenant's access permissions for a specific data row through bitmap lookup and simple bit operations, greatly simplifying the verification process and reducing computational complexity. In practical applications, the linear relationship between the size of the bitmap and the number of data rows in the target data block indicates that the storage overhead of the bitmap is predictable and efficient. It is only related to the number of rows contained in the target data block. For example, if a target data block has N rows, the bitmap may require N bits (or N / 8 bytes) to represent it. The fact that it doesn't grow with the size of the target data block further emphasizes this efficiency; even if a single row of data in the target data block is very large, the size of the bitmap will not increase accordingly, thus effectively controlling the use of storage resources.
[0115] This application's solution pre-computes and encodes complex access permission rules into row-level access permission bitmaps, transforming complex logical judgments that might otherwise require real-time processing into efficient bitmap lookups and bit operations. When a data access request is received, the system no longer needs to traverse and evaluate multiple access permission rules; instead, it directly uses the prepared bitmap for rapid verification. This mechanism makes the computational load of access permission verification linearly related to the number of data rows and independent of the specific size of the data rows, thus significantly reducing the computational burden for each request and improving the efficiency and response speed of access permission verification.
[0116] In some preferred embodiments, suppose a target data block contains 10,000 rows of data, and complex access permission rules are defined for these rows, such as based on multiple dimensions like user role, data sensitivity, and time range. Without a pre-computed bitmap, each access request might require evaluating these rules for each of the 10,000 rows of data, which would be a time-consuming process.
[0117] However, according to the scheme of this application, when these access permission rules are defined or updated, the system immediately performs batch processing to generate a 10,000-bit row-level access permission bitmap for these 10,000 rows of data. Each bit in the bitmap (e.g., the i-th bit) represents whether the i-th row of data is allowed to be accessed by a specific tenant. For example, if the 500th bit of the bitmap is 1, it means that the 500th row of data is allowed to be accessed; if it is 0, it means that access is denied.
[0118] When a data access request arrives, the system no longer needs to recalculate these complex rules. It simply needs to determine whether the requester has access rights to the target data row based on the requester's tenant identification information, either by direct querying or through simple bitwise operations (such as bitwise AND). For example, if a bitwise AND operation is performed between the requester's tenant's permission bitmap and the target data block's row-level access permission bitmap, a 1 in the resulting bitmap indicates that the row is accessible. This approach transforms complex logical judgments into efficient bitwise operations, completing access permission checks within milliseconds, significantly improving data access response speed and system throughput. Furthermore, this 10,000-bit bitmap occupies only about 1.25KB of storage space (10,000 bits / 8 bits / byte), and its size is independent of the actual content stored in each row, demonstrating highly efficient storage utilization.
[0119] refer to Figure 2 , Figure 2 This is a schematic diagram of a data security isolation system based on access permission verification provided in an embodiment of the present invention, applied in a multi-tenant distributed computing platform, including: The data access request receiving module is used to receive a data access request and parse the requesting tenant identification information and the requested data range from the data access request; A data block acquisition module is used to acquire a target data block, wherein the target data block is configured with a row-level access permission bitmap and access permission version information indicating the basis for the generation of the row-level access permission bitmap; The version information acquisition module is used to obtain the latest access permission version information; The version comparison module is used to compare the access permission version information of the target data block with the latest access permission version information to determine whether the access permission version of the target data block is outdated. The difference identification module is used to identify the difference information between the latest access permission and the access permission on which the target data block is based when the access permission version of the target data block is outdated. The correction information generation module is used to perform batch permission comparison on the data rows in the target data block based on the difference information, and generate row-level access permission correction information; The bitmap update module is used to logically combine the row-level access permission correction information with the original row-level access permission bitmap in the target data block, and update the row-level access permission bitmap of the target data block in place. The permission verification module is used to perform row-level permission verification based on the updated row-level permission bitmap and the requesting tenant identification information; The access response module is used to return data within the requested data range when the verification passes, and to return a rejection message when the verification fails.
[0120] The system proposed in this application aims to solve the problem of row-level data security isolation failure caused by dynamic changes in permission rules and reuse of intermediate data caches in multi-tenant distributed computing platforms. This system achieves dynamic and efficient management of data block access permissions by introducing a row-level access permission bitmap and access permission version information, and designing a bitmap update mechanism based on version comparison and difference correction.
[0121] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.
Claims
1. A data security isolation method based on access permission verification, applied in a multi-tenant distributed computing platform, characterized in that, include: Receive a data access request, and parse the requesting tenant identification information and the requested data range from the data access request; Obtain a target data block, wherein the target data block is configured with a row-level access permission bitmap and access permission version information indicating the basis for generating the row-level access permission bitmap; Get the latest access permission version information; Compare the access permission version information of the target data block with the latest access permission version information to determine whether the access permission version of the target data block is outdated; When the access permission version of the target data block is outdated, identify the difference between the latest access permission and the access permission on which the target data block is based; Based on the difference information, batch processing permission comparison is performed on the data rows in the target data block to generate row-level access permission correction information; The row-level access permission correction information is logically combined with the original row-level access permission bitmap in the target data block to update the row-level access permission bitmap of the target data block in place. Based on the updated row-level access permission bitmap and the requesting tenant identification information, perform row-level access permission verification; If the verification passes, return the data within the requested data range; If the verification fails, a rejection message is returned.
2. The method according to claim 1, characterized in that, The step of logically combining the row-level access permission correction information with the original row-level access permission bitmap in the target data block to update the row-level access permission bitmap of the target data block in situ includes: Obtain the latest access permissions and the access permissions on which the target data block is based; Identify the newly added denial conditions in the latest access permissions relative to the access permissions on which the target data block is based, and form incremental tightening conditions; Identify the rejection conditions for removing the access permissions relative to the target data block in the latest access permissions, and form incremental relaxation conditions; Based on the incremental tightening conditions, a batch processing permission comparison is performed on the data rows in the target data block to generate a tightening bitmap. Based on the incremental relaxation conditions, a batch permission comparison is performed on the data rows in the target data block to generate a relaxation bitmap. Perform a bitwise AND operation between the original row-level access permission bitmap in the target data block and the tightened bitmap to obtain an intermediate access permission bitmap; Perform a bitwise OR operation between the intermediate permission bitmap and the relaxed bitmap to obtain the final row-level access permission bitmap; The final row-level access permission bitmap is updated in-place to the target data block.
3. The method according to claim 2, characterized in that, The step of performing a bitwise AND operation between the original row-level access permission bitmap in the target data block and the tightened bitmap to obtain an intermediate access permission bitmap includes: Add the bitmap operation requests to be executed to the bitmap operation queue; Acquire the mutex lock corresponding to the target data block; Read the row-level access permission bitmap of the target data block; Perform a bitwise AND operation between the row-level access permission bitmap and the tightened bitmap to obtain an intermediate access permission bitmap; Write the intermediate permission bitmap into the bitmap area of the target data block; Release the mutex.
4. The method according to claim 2, characterized in that, The step of performing a bitwise OR operation between the intermediate permission bitmap and the relaxed bitmap to obtain the final row-level access permission bitmap includes: Acquire the bitmap operation lock corresponding to the target data block; Read the row-level access permission bitmap of the target data block; Perform a bitwise OR operation between the row-level access permission bitmap and the relaxation bitmap to obtain the final row-level access permission bitmap; Update the final row-level access permission bitmap in-place to the target data block; Release the bitmap operation lock.
5. The method according to claim 1, characterized in that, The step of obtaining the target data block, wherein the target data block is configured with a row-level access permission bitmap and access permission version information indicating the basis for generating the row-level access permission bitmap, includes: When a computing node generates or receives a target data block, it obtains the globally unique identifier and access permission metadata header of the target data block. Submit the globally unique identifier and the access permission metadata header to the consistency coordination service; The consistency coordination service maintains a consistent view of data block metadata, receives access permission update broadcasts, and updates the latest access permission version number maintained internally. When the target data block is replicated or cached between nodes, the receiving node initiates a query request to the consistency coordination service to obtain the latest access permission version number of the target data block and the corresponding row-level access permission bitmap. The consistency coordination service compares the version number currently carried by the target data block with the latest access permission version number maintained internally. If the version number currently carried by the target data block is outdated, the consistency coordination service performs differential bitmap correction logic on the metadata of the target data block to generate a corrected row-level access permission bitmap and the latest version number; The consistency coordination service returns the corrected row-level access permission bitmap and the latest version number to the receiving node; The receiving node writes the corrected metadata in-place into the access permission metadata header of the target data block.
6. The method according to claim 5, characterized in that, The step of obtaining the globally unique identifier and access permission metadata header of the target data block when a computing node generates or receives a target data block includes: When a computing node generates a target data block, it obtains the globally unique identifier of the target data block from the globally unique identifier generator; Bind the globally unique identifier to the access permission metadata header of the target data block; When a receiving node receives a target data block, it verifies the validity of the globally unique identifier carried by the target data block. If the verification passes, the globally unique identifier and the access permission metadata header are submitted to the consistency coordination service; The globally unique identifier is encoded with the access permission version information on which the target data block is based; the consistency coordination service directly indexes the latest access permission version and differential bitmap corresponding to the globally unique identifier.
7. The method according to claim 6, characterized in that, The step of obtaining the globally unique identifier of the target data block from the globally unique identifier generator when a computing node generates a target data block includes: When a compute node needs to generate a globally unique identifier for a target data block, it sends a request to the master identifier generation service. If the primary identifier generation service does not return an identifier within a preset response time, the computing node switches to the local backup identifier generation module. The backup identifier generation module generates a temporary identifier based on the machine identification information of the computing node, the current timestamp, and the incrementing local sequence number; The temporary identifier carries a tag indicating that it was generated for backup; When the primary identifier generation service is restored to availability, the compute node submits all temporary identifiers and corresponding data block information generated by the backup identifier generation module to the primary identifier generation service to perform registration and global uniqueness verification. If the primary identifier generation service detects a conflict in the temporary identifier, it reassigns a globally unique identifier to the conflicting data block and notifies the computing node of the updated identifier.
8. The method according to claim 6, characterized in that, The step of obtaining the globally unique identifier of the target data block from the globally unique identifier generator when a computing node generates a target data block includes: When a compute node needs to generate a globally unique identifier for a target data block, it obtains the tenant identification information of the current operation. Submit the tenant identification information to the globally unique identifier generator; The globally unique identifier generator receives the tenant identification information and obtains the corresponding tenant-specific identifier segment from the preset tenant identifier allocation rules according to the tenant identification information; The globally unique identifier generator combines the current time information and the incrementing sequence number within the tenant-specific identifier segment to generate a globally unique identifier for the target data block. The globally unique identifier generator returns the globally unique identifier of the generated target data block to the computing node.
9. The method according to claim 1, characterized in that, The row-level access permission verification is performed based on the pre-computed row-level access permission bitmap, without the need to calculate access permission rules in real time at the time of request; the size of the bitmap is linearly related to the number of data rows in the target data block, and does not increase with the content size of the target data block.
10. A data security isolation system based on access permission verification, applied in a multi-tenant distributed computing platform, characterized in that, include: The data access request receiving module is used to receive a data access request and parse the requesting tenant identification information and the requested data range from the data access request; A data block acquisition module is used to acquire a target data block, wherein the target data block is configured with a row-level access permission bitmap and access permission version information indicating the basis for the generation of the row-level access permission bitmap; The version information acquisition module is used to obtain the latest access permission version information; The version comparison module is used to compare the access permission version information of the target data block with the latest access permission version information to determine whether the access permission version of the target data block is outdated. The difference identification module is used to identify the difference information between the latest access permission and the access permission on which the target data block is based when the access permission version of the target data block is outdated. The correction information generation module is used to perform batch permission comparison on the data rows in the target data block based on the difference information, and generate row-level access permission correction information; The bitmap update module is used to logically combine the row-level access permission correction information with the original row-level access permission bitmap in the target data block, and update the row-level access permission bitmap of the target data block in place. The permission verification module is used to perform row-level permission verification based on the updated row-level permission bitmap and the requesting tenant identification information; The access response module is used to return data within the requested data range when the verification passes, and to return a rejection message when the verification fails.
